# GPT-J weights on HuggingFace

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-j-weights-on-huggingface%2F10922%3E%5C
**Category:** Models\
**Created:** [October 20, 2021, 6:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-j-weights-on-huggingface%2F10922 "2021-10-20T06:34:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vj1148](https://avatars.discourse-cdn.com/v4/letter/v/67e7ee/32.png) [@vj1148](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvj1148)%5C
**Post date:** [October 20, 2021, 6:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-j-weights-on-huggingface%2F10922%2F1 "2021-10-20T06:34:05Z")

</div>

The GPT-J model which is available on HuggingFace - is this a full weight model or slim weight? Any idea?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [October 20, 2021, 7:32am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-j-weights-on-huggingface%2F10922%2F2 "2021-10-20T07:32:47Z")

</div>

You can use both. The default one is the full version (float32). To use the slim version, you can do the following:

```auto
from transformers import GPTJForCausalLM
import torch

model = GPTJForCausalLM.from_pretrained("EleutherAI/gpt-j-6B", revision="float16", torch_dtype=torch.float16, low_cpu_mem_usage=True)

```

---

<div class="post-metadata">

**Author:** ![vj1148](https://avatars.discourse-cdn.com/v4/letter/v/67e7ee/32.png) [@vj1148](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvj1148)%5C
**Post date:** [October 20, 2021, 12:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-j-weights-on-huggingface%2F10922%2F3 "2021-10-20T12:42:27Z")

</div>

Sure. Thanks for the help.
