# HF Datasets best practices

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhf-datasets-best-practices%2F58566%3E%5C
**Category:** 🤗Datasets\
**Created:** [October 14, 2023, 6:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhf-datasets-best-practices%2F58566 "2023-10-14T18:04:05Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![lucmos](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lucmos/32/20356_2.png) [@lucmos](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flucmos)%5C
**Post date:** [October 14, 2023, 6:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhf-datasets-best-practices%2F58566%2F1 "2023-10-14T18:04:05Z")

</div>

Hello everyone!

We are building a [generic cookiecutter template to bootstrap PyTorch projects](https://github.com/grok-ai/nn-template) and to avoid writing boilerplate.

Among the other things, we integrated the HF Datasets library.

This means that any new project generated with this template already has a hugging face dataset configured and ready to be used to train a neural network 😄

I would really appreciate any feedback on how to improve this integration – are there any best practices that we are not using?

The interesting files are the following:

- `.../data/datamodule.py` where the (torchvision) transforms are applied with `set_transform`; and the DataLoaders are created.
- `.../utils/hf_io.py` that handle the loading of the train/val/test splits and some basic pre-processing.

I am particular interested in speed related improvements

Thanks!