# Question about validation and testing loss
**URL:**
**Author:**  [@Cheltone](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FCheltone)%5C
**Post date:** [April 17, 2022, 10:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-validation-and-testing-loss%2F16910%2F1 "2022-04-17T22:25:06Z")
Hello,
Just double checking my numbers here. Would these numbers for testing and validation be indicative of overfitting? I feel like I might be on the verge but would very much appreciate a second opinion.
Thanks!
| Training Loss | Epoch | Step | Validation Loss | Precision | Accuracy | F1 |
| --- | --- | --- | --- | --- | --- | --- |
| 0.5894 | 0.4 | 500 | 0.4710 | 0.8381 | 0.7747 | 0.7584 |
| 0.3863 | 0.8 | 1000 | 0.3000 | 0.8226 | 0.8737 | 0.8858 |
| 0.2272 | 1.2 | 1500 | 0.1973 | 0.9593 | 0.9333 | 0.9329 |
| 0.1639 | 1.6 | 2000 | 0.1694 | 0.9067 | 0.9367 | 0.9403 |
| 0.1263 | 2.0 | 2500 | 0.1128 | 0.9657 | 0.9597 | 0.9603 |
| 0.0753 | 2.4 | 3000 | 0.1305 | 0.9614 | 0.967 | 0.9679 |
| 0.0619 | 2.8 | 3500 | 0.1246 | 0.9633 | 0.9697 | 0.9705 |
---
**Author:**  [@infinitejoy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Finfinitejoy)%5C
**Post date:** [April 18, 2022, 3:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-validation-and-testing-loss%2F16910%2F2 "2022-04-18T03:15:37Z")
More than validation loss this might be the case where there is an overlap in the training and validation data.
Go through the validation data carefully and check if the data has somehow crept into the training data.
---
**Author:**  [@Cheltone](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FCheltone)%5C
**Post date:** [April 18, 2022, 3:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-validation-and-testing-loss%2F16910%2F3 "2022-04-18T03:34:43Z")
Thanks for commenting and itโs nice to meet you! Iโm positive my testing/validation data were defined. I augmented my data set with back-translation. I removed my duplicates but maybe that could be it? Would you mind explaining your thoughts a bit more?
Cheers!
---
**Author:**  [@infinitejoy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Finfinitejoy)%5C
**Post date:** [April 18, 2022, 3:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-validation-and-testing-loss%2F16910%2F4 "2022-04-18T03:46:59Z")
Ok. If you are using data augmentation then you need to make sure two things.
1. Do the augmentation after the train test split. That way you make sure that train data does not accidentally creep into the test set.
2. Only run the augmentation on the training data.
Here is a reference to this [python - Data augmentation before splitting - Stack Overflow](https://stackoverflow.com/a/50639857/5417164).
Now that I think about it this can mean overfitting as well. The validation results look too good. You can try one more thing which is increasing the epochs. If the F1 still keeps on decreasing that would mean a high chance of overfitting. Please relook into the way you are using the augmented data.
---
**Author:**  [@Cheltone](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FCheltone)%5C
**Post date:** [April 18, 2022, 3:58am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-validation-and-testing-loss%2F16910%2F5 "2022-04-18T03:58:16Z")
Ok great! The accuracy and f1 both stagnate after 3 epochs. Iโll read up. Thanks for the ideas ๐
---
**Author:**  [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [April 19, 2022, 6:52am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-validation-and-testing-loss%2F16910%2F6 "2022-04-19T06:52:28Z")
Just fyi: we typically make a distinction between a test set and a validation set. They are not the same thing. A validation (or development) set is used during training to intermediately evaluate the model, to probe the current performance every x steps/epochs. This way we can quickly see if something is wrong (data pollution, overfitting). A test set is a hold-out set that we only ever use _after_ training (not during training, not even for evaluation, also not for cross-validation training or hyperparameter search). It should be explicitly be _held out_ to ensure that the trained model is not biased towards it at all.
Tl;dr: validation/dev set: to monitor training; test set: hold-out for final evaluation (yes, it is confusing)
---
**Author:**  [@Cheltone](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FCheltone)%5C
**Post date:** [April 19, 2022, 1:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-validation-and-testing-loss%2F16910%2F7 "2022-04-19T13:23:21Z")
Thanks for mentioning it. It was a typing accident lol. Nonetheless, I appreciate your explanation ๐