# Add data augmentation process during training every epoch
**URL:**
**Author:**  [@yusukemori](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fyusukemori)%5C
**Post date:** [January 19, 2021, 5:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadd-data-augmentation-process-during-training-every-epoch%2F3274%2F1 "2021-01-19T05:46:55Z")
Hello,
I’d like to process my training dataset every epoch.
I want to add random processing as data augmentation, and I want to do it during training, not preprocessing.
I think I can do it with `Trainer`, `DataCollator`, or ` __getitem__ ` of `datasets.arrow_dataset.Dataset`, but where should I do it?
For the evaluation set and test set, I plan to do a preprocess using `datasets.arrow_dataset.Dataset.map`.
Thank you in advance.
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [January 19, 2021, 9:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadd-data-augmentation-process-during-training-every-epoch%2F3274%2F2 "2021-01-19T21:14:07Z")
The `DataCollator` can help if you have something randomized in the call that returns the batch. A `getitem` in your Dataset can also help, it all depends on what you are trying to do exactly.
The `Trainer` in itself has nothing implemented for data augmentation, so it won’t help you.
---
**Author:**  [@yusukemori](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fyusukemori)%5C
**Post date:** [January 20, 2021, 3:52am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadd-data-augmentation-process-during-training-every-epoch%2F3274%2F3 "2021-01-20T03:52:24Z")
Hi @sgugger
Thank you for the answer!
I now understand for my purpose I should treat a `DataCollator` or `getitem` in a Dataset, depending on what I am trying to do, and the `Trainer` is not an appropriate place in which data augmentation is done.
Referring to your answer in my another question [How to use Seq2SeqTrainer (Seq2SeqDataCollator) in v4.2.1 - #5 by sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-seq2seqtrainer-seq2seqdatacollator-in-v4-2-1%2F3243%2F5)%2C
I’ll consider the best way to implement to get something randomized in my case.
Thank you so much!