# Construct batch with token numbers

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconstruct-batch-with-token-numbers%2F15604%3E%5C
**Category:** Beginners\
**Created:** [March 11, 2022, 9:30am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconstruct-batch-with-token-numbers%2F15604 "2022-03-11T09:30:15Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![MACong](https://avatars.discourse-cdn.com/v4/letter/m/f04885/32.png) [@MACong](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMACong)%5C
**Post date:** [March 11, 2022, 9:30am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconstruct-batch-with-token-numbers%2F15604%2F1 "2022-03-11T09:30:15Z")

</div>

The default batch size in huggingface’s [TrainingArguments](https://huggingface.co/docs/transformers/v4.15.0/en/main_classes/trainer#transformers.TrainingArguments.per_device_train_batch_size) is 8 samples. However, when I finetune models like T5 on machine translation task, batch size of fixed number of samples are not efficient for GPU utilization.

I wonder whether Huggingface/datasets has the batch construction method by setting token numbers. For example, if we set the batch size of 3000 tokens, then the batch would have several samples, and the total number of tokens are around 3000.

Is there anyone can help me?  
Emergency!

Thanks a lot ~!

---

<div class="post-metadata">

**Author:** ![anwarika](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@anwarika](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanwarika)%5C
**Post date:** [March 11, 2022, 2:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconstruct-batch-with-token-numbers%2F15604%2F2 "2022-03-11T14:00:06Z")

</div>

I don’t think I understand your inquiry? Datasets you can set a max\_token size