# Can't load fine tuned LLamav2 7b
**URL:**
**Author:**  [@Stoemb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FStoemb)%5C
**Post date:** [July 29, 2023, 12:25am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-load-fine-tuned-llamav2-7b%2F48573%2F1 "2023-07-29T00:25:05Z")
I fined tuned a Llama 2 7b model and uploaded it to hugging face but now when will load it in google colab I ran out of system ram. (fine tuned model: Stoemb/llama-2-7b-html2text)
I loaded the model as followed:
model\_name = “Stoemb/llama-2-7b-html2text”
bnb\_config = BitsAndBytesConfig(
load\_in\_4bit=True,
bnb\_4bit\_quant\_type=“nf4”,
bnb\_4bit\_compute\_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from\_pretrained(
model\_name,
quantization\_config=bnb\_config,
trust\_remote\_code=True
)
model.config.use\_cache = False
---
**Author:**  [@Remmet](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRemmet)%5C
**Post date:** [July 30, 2023, 2:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-load-fine-tuned-llamav2-7b%2F48573%2F2 "2023-07-30T02:44:47Z")
I’m still learning myself but i have been playing with a Llama 2 7B model in free Colab and I’ve found i need to set the device\_map to “auto” as well as loading in 4 or 8 bit.
---
**Author:**  [@dannstrom](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdannstrom)%5C
**Post date:** [October 13, 2023, 10:40am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-load-fine-tuned-llamav2-7b%2F48573%2F3 "2023-10-13T10:40:56Z")
Don’t know if this is still an issue for you, but I had the same problem first time I fine-tuned the llama2 and tried to reload it.
The problem is that the default shard size when pushed to hub is 10GB, which is too much for the T4 GPU.
You can read more here:
> **[Instantiating a big model](https://huggingface.co/docs/transformers/big_models#sharded-checkpoints)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.
To solve this you have to specify a smaller shard size as in the example below.
```auto
!huggingface-cli login
model.push_to_hub(your_model_name, max_shard_size='2GB')
tokenizer.push_to_hub(your_model_name)
```
This solved the problem I had at least, which sounds a lot similar to yours.