# Fine-tunening a multimodal model

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tunening-a-multimodal-model%2F77491%3E%5C
**Category:** Beginners\
**Created:** [March 15, 2024, 3:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tunening-a-multimodal-model%2F77491 "2024-03-15T03:01:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![PatchouliPatch](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/patchoulipatch/32/24878_2.png) [@PatchouliPatch](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPatchouliPatch)%5C
**Post date:** [March 15, 2024, 3:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tunening-a-multimodal-model%2F77491%2F1 "2024-03-15T03:01:46Z")

</div>

hello! Does anyone have an example on how to fine-tune a multimodal model? The examples that I found only have texts as input, i.e. LLMs. I’m dealing with Vision-Language Models

---

<div class="post-metadata">

**Author:** ![soledadalborno](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/soledadalborno/32/26752_2.png) [@soledadalborno](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsoledadalborno)%5C
**Post date:** [May 8, 2024, 1:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tunening-a-multimodal-model%2F77491%2F2 "2024-05-08T01:48:48Z")

</div>

Did you find one?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [May 8, 2024, 7:43am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tunening-a-multimodal-model%2F77491%2F3 "2024-05-08T07:43:50Z")

</div>

Hi,

Yes find them all in my repository [GitHub - NielsRogge/Transformers-Tutorials: This repository contains demos I made with the Transformers library by HuggingFace.](https://github.com/NielsRogge/Transformers-Tutorials/tree/master).

Demo notebooks are grouped per model, so I’d recommend taking a look at the BLIP-2, Idefics2, PaliGemma and LLaVa folders for some examples.

---

<div class="post-metadata">

**Author:** ![ritabratamaiti](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ritabratamaiti/32/35791_2.png) [@ritabratamaiti](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fritabratamaiti)%5C
**Post date:** [November 18, 2024, 10:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tunening-a-multimodal-model%2F77491%2F4 "2024-11-18T10:53:34Z")

</div>

Hi! You can try AnyModal: [GitHub - ritabratamaiti/AnyModal: AnyModal is a Flexible Multimodal Language Model Framework](https://github.com/ritabratamaiti/AnyModal)  
It’s a project I have been working on that allows for the creation of multimodal LLMs.

---

<div class="post-metadata">

**Author:** ![Ismail-IFAKIR](https://avatars.discourse-cdn.com/v4/letter/i/13edae/32.png) [@Ismail-IFAKIR](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FIsmail-IFAKIR)%5C
**Post date:** [December 25, 2024, 12:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tunening-a-multimodal-model%2F77491%2F5 "2024-12-25T12:17:36Z")

</div>

Hello guys,  
I hope you are doing well, how can fine tune Multimodal LLM like (LLaVA, PaliGemma …) for Multimodal Aspect Based Sentiment Analysis (Text and Image)