# Inference slower after fine tuning

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-slower-after-fine-tuning%2F73153%3E%5C
**Category:** Beginners\
**Created:** [February 14, 2024, 12:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-slower-after-fine-tuning%2F73153 "2024-02-14T12:25:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jlamon](https://avatars.discourse-cdn.com/v4/letter/j/a9adbd/32.png) [@jlamon](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjlamon)%5C
**Post date:** [February 14, 2024, 12:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-slower-after-fine-tuning%2F73153%2F1 "2024-02-14T12:25:27Z")

</div>

Dear,

Before fine tuning the model, it takes 2 minutes to get the predictions on the dataset. However, after training, it takes around 12 minutes (same test set).

Does anybody have a reason to this issue ? Setting the model to cuda gives an error.

Notes: I’m using SFTTrainer and QLoRA.

I already thank you for your answer.

---

<div class="post-metadata">

**Author:** ![dblakely](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@dblakely](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdblakely)%5C
**Post date:** [February 14, 2024, 5:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-slower-after-fine-tuning%2F73153%2F2 "2024-02-14T17:40:33Z")

</div>

Does the finetuned model write a lot more than the base model?

---

<div class="post-metadata">

**Author:** ![jlamon](https://avatars.discourse-cdn.com/v4/letter/j/a9adbd/32.png) [@jlamon](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjlamon)%5C
**Post date:** [February 14, 2024, 5:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-slower-after-fine-tuning%2F73153%2F3 "2024-02-14T17:54:32Z")

</div>

Hey, no it does not.

I’m creating a classification task through prompts. Hence it only returns a few words (being the label’s name).
