# Load Pascal VOC with different configurations from s3

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-pascal-voc-with-different-configurations-from-s3%2F69363%3E%5C
**Category:** 🤗Datasets\
**Created:** [January 16, 2024, 12:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-pascal-voc-with-different-configurations-from-s3%2F69363 "2024-01-16T12:41:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Revist](https://avatars.discourse-cdn.com/v4/letter/r/f19dbf/32.png) [@Revist](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRevist)%5C
**Post date:** [January 16, 2024, 12:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-pascal-voc-with-different-configurations-from-s3%2F69363%2F1 "2024-01-16T12:41:00Z")

</div>

Hello everybody,

I would like to use s3 as storage for a dataset with different configurations. A good example is Pascal VOC dataset here [fuliucansheng/pascal\_voc at main](https://huggingface.co/datasets/fuliucansheng/pascal_voc/tree/main)  
The underlying data of all configurations is the same and main difference is how \_generate\_examples function is executed. Local caching works seems to work well and reuse the images under different configs.

Now I would like to be able to specify dataset + configuration and load it directly from s3. What is the best way to do that?

I tried

```auto
dataset.save_to_disk("s3:/xxx/xxx", storage_options=storage_options)
dataset = load_from_disk("s3:/xxx/xxx", storage_options=storage_options)

```

but that works for a single configuration apparently.

---

<div class="post-metadata">

**Author:** ![Revist](https://avatars.discourse-cdn.com/v4/letter/r/f19dbf/32.png) [@Revist](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRevist)%5C
**Post date:** [January 17, 2024, 5:24pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-pascal-voc-with-different-configurations-from-s3%2F69363%2F2 "2024-01-17T17:24:32Z")

</div>

Hmm it seems that what I am asking for is basically what hf hub offers and could be achieved by creating a version of the class here [datasets/src/datasets/load.py at main · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/blob/main/src/datasets/load.py#L1394) but for s3.