# Trade off between max\_length vs loss

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrade-off-between-max-length-vs-loss%2F37513%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 23, 2023, 6:37pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrade-off-between-max-length-vs-loss%2F37513 "2023-04-23T18:37:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![agneet](https://avatars.discourse-cdn.com/v4/letter/a/958977/32.png) [@agneet](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fagneet)%5C
**Post date:** [April 23, 2023, 6:37pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrade-off-between-max-length-vs-loss%2F37513%2F1 "2023-04-23T18:37:43Z")

</div>

Hello, I am fine-tuning GPT-J-6B with each input data point = 2048 tokens. My loss decreases, albeit at a very slow rate. I was hoping to understand if the size of 2048 might be an issue? Is there a correlation between max. size and the loss? Should I decrease the max. size to a lower number (ex. 512) and try again?