# T5 Finetuning Tips

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-finetuning-tips%2F684%3E%5C
**Category:** Models\
**Created:** [August 11, 2020, 8:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-finetuning-tips%2F684 "2020-08-11T20:39:06Z")\
**Posts on this page:** 1\
**Showing post:** 29

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [January 12, 2021, 12:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-finetuning-tips%2F684%2F29 "2021-01-12T12:11:08Z")

</div>

Hey there, you guys might find this interesting, we have just fixed the fp16 issue for some of the T5 models

More details below

> [@T5 fp16 issue is fixed](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-fp16-issue-is-fixed%2F3139)%3A
>
> We have just fixed the T5 fp16 issue for some of the T5 models! (Announcing it here, since lots of users were facing this issue and T5 is one most widely used model in the library) TL;DR: Previously, there was an issue when using T5 models in fp16; it was producing nan loss and logits. Now on the master, this issue is fixed for the following T5 models and versions. Now you should be able to train and use these models for inference in fp16 and see a decent speed-up! T5v1 : t5-small , t5-…

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-finetuning-tips%2F684)._