# Merging in successive loras to a base model

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmerging-in-successive-loras-to-a-base-model%2F110523%3E%5C
**Category:** Beginners\
**Created:** [October 7, 2024, 1:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmerging-in-successive-loras-to-a-base-model%2F110523 "2024-10-07T01:06:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aalleavitch](https://avatars.discourse-cdn.com/v4/letter/a/c89c15/32.png) [@aalleavitch](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faalleavitch)%5C
**Post date:** [October 7, 2024, 1:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmerging-in-successive-loras-to-a-base-model%2F110523%2F1 "2024-10-07T01:06:56Z")

</div>

I have multiple loras that I’ve trained on a base model and I’ve been attempting to successively merge them into the base model, however there appears to be some issue with the resulting model. Specifically, I believe the issue is with the tokenizer, but I’m uncertain how to fix this issue, as my understanding is that the tokenizer should simply be the same as the base model.

When I load the base model into text-generation-webui and apply all three loras, I get good generations with the expected behavior, however, after merging them together with the below code I get gibberish that seems very clearly to be applying incorrect tokens.

```auto

model = AutoModelForCausalLM.from_pretrained(
    "unsloth/Mistral-Small-Instruct-2409-bnb-4bit"
)
tokenizer = AutoTokenizer.from_pretrained(
    "unsloth/Mistral-Small-Instruct-2409-bnb-4bit"
)

model = PeftModel.from_pretrained(model, "hobie-lora4")
model = model.merge_and_unload()
model.save_pretrained("HobieLLMPart4")
tokenizer.save_pretrained("HobieLLMPart4")
model = PeftModel.from_pretrained(model, "hobie-lora5")
model = model.merge_and_unload()
model.save_pretrained("HobieLLMPart5")
tokenizer.save_pretrained("HobieLLMPart5")
model = PeftModel.from_pretrained(model, "hobie-lora6")
model = model.merge_and_unload()
model.save_pretrained("HobieLLMPart6")
tokenizer.save_pretrained("HobieLLMPart6")

```

---

<div class="post-metadata">

**Author:** ![Chahnwoo](https://avatars.discourse-cdn.com/v4/letter/c/a88e57/32.png) [@Chahnwoo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FChahnwoo)%5C
**Post date:** [October 7, 2024, 5:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmerging-in-successive-loras-to-a-base-model%2F110523%2F2 "2024-10-07T05:02:04Z")

</div>

I’ve never tried this myself, so this may not be entirely correct, but it seems to me that while the first merge (with `hobie-lora4`) would work, the subsequent merges would not.

The `PeftModel.from_pretrained(model, directory)` function may not be working as expected because of the type differences between models initialized with the `AutoModelForCausalLM.from_pretrained()` function and models resulting from the `model.merge_and_unload()` function.

---

<div class="post-metadata">

**Author:** ![BenjaminB](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/benjaminb/32/30898_2.png) [@BenjaminB](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBenjaminB)%5C
**Post date:** [October 25, 2024, 1:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmerging-in-successive-loras-to-a-base-model%2F110523%2F3 "2024-10-25T13:07:22Z")

</div>

If the different LoRA adapters were all trained on the base model, it would not be surprising that you cannot just merge all of them together. After you merge the first adapter, the new base model (i.e. old base model + first LoRA) is different from what the 2nd adapter was trained on, so it doesn’t work as expected.

So either, you need to train the 2nd LoRA based on the merged 1st LoRA, etc. Or you try the `add_weighted_adapter` method, which combines multiple LoRA adapters into a single one: [LoRA](https://huggingface.co/docs/peft/developer_guides/lora#merge-lora-weights-into-the-base-model).
