# Prompt-tuning for Multimodal model
**URL:**
**Author:**  [@alphairawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falphairawan)%5C
**Post date:** [March 25, 2025, 9:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprompt-tuning-for-multimodal-model%2F147407%2F1 "2025-03-25T21:21:40Z")
I am currently working on multimodal model, llava-next-video for video classification. I would like to try prompt tuning on that model. When I run through [this notebook](https://colab.research.google.com/github/huggingface/cookbook/blob/main/notebooks/en/prompt_tuning_peft.ipynb#scrollTo=-4jd3zCGVaTG) example for prompt tuning and the [documentation](https://huggingface.co/docs/peft/main/en/package_reference/prompt_tuning), I did not find specification for the prompt data for multimodal prompting. In my case, I use following chat template,
```auto
template= [
{
"role": "user",
"content": [
{"type": "video"},
{
"type": "text",
"text": (
"Please classify the behaviour in the video if it contain punching"
)
}
]
}
]
```
Is there any reference or github repo on how to use peft for prompt tuning to multimodal prompting using chat template?
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 26, 2025, 8:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fprompt-tuning-for-multimodal-model%2F147407%2F2 "2025-03-26T08:23:48Z")
Perhaps this is an example.
>