# Not able to overfit a transformer model on my data

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fnot-able-to-overfit-a-transformer-model-on-my-data%2F43245%3E%5C
**Category:** 🤗Transformers\
**Created:** [June 14, 2023, 2:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fnot-able-to-overfit-a-transformer-model-on-my-data%2F43245 "2023-06-14T14:57:36Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![SalehGh17](https://avatars.discourse-cdn.com/v4/letter/s/b9e5f3/32.png) [@SalehGh17](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSalehGh17)%5C
**Post date:** [June 14, 2023, 2:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fnot-able-to-overfit-a-transformer-model-on-my-data%2F43245%2F1 "2023-06-14T14:57:37Z")

</div>

I have a very bursty timeseries for the forecasting purpose.  
purposely I am trying to overfit on a small subset of data. I took 30 sample trajectories of length 150 to train the transformer model on. I am using a transformer with more than 10Million parameters, however the model doesn’t overrfit.  
data.shape=30,150  
in each trajectory the first 75 points will be shown to the model and the model should predict the next 75 steps (75 step ahead forecasting). I am using adam optimizer and mse loss  
Do you have any idea why I can’t overfit?
