# How does the finetune on transformer (t5) work

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-does-the-finetune-on-transformer-t5-work%2F16585%3E%5C
**Category:** Beginners\
**Created:** [April 7, 2022, 2:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-does-the-finetune-on-transformer-t5-work%2F16585 "2022-04-07T14:42:30Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![NimaBoscarino](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nimaboscarino/32/4358_2.png) [@NimaBoscarino](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FNimaBoscarino)%5C
**Post date:** [April 11, 2022, 5:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-does-the-finetune-on-transformer-t5-work%2F16585%2F4 "2022-04-11T17:29:40Z")

</div>

Ah I’m definitely not the right person to answer this, but I _think_ you should be able to just alter the `model.lm_head` to something like:

```auto
model.lm_head = nn.Sequential(
    nn.Linear(in_features=model.lm_head.in_features, out_features=<SOMENUMBER>, bias=False),
    nn.Linear(in_features=<SOMENUMBER>, out_features=model.lm_head.in_features, bias=False)
)

```

And you can add other stuff in there too I think, as long the final output layer matches the expected dimensions (unless you want to change that too [How do I change the classification head of a model? - #19 by nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-do-i-change-the-classification-head-of-a-model%2F4720%2F19))

If you’re just adding layers to the head I don’t think that you need edit the source code. If you needed to change stuff within the network, e.g. [to make changes to the T5Block](https://github.com/huggingface/transformers/issues/7360), I think that’s when you would dive into the source code.

I’m curious to learn more about this though, so do update this post with anything new that you learn about this!

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-does-the-finetune-on-transformer-t5-work%2F16585)._
