# Help creating dataset from s3 bucket with parquet files
**URL:**
**Author:**  [@jjovalle99](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjjovalle99)%5C
**Post date:** [July 27, 2023, 5:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhelp-creating-dataset-from-s3-bucket-with-parquet-files%2F48429%2F1 "2023-07-27T17:38:11Z")
Hi! I have a bunch of parquet files with the same schema in a S3 bucket. Is there a way to load a dataset using `load_dataset`?
I’m doing this but is not working:
```auto
!pip install -q -U git+https://github.com/huggingface/transformers.git
!pip install -q -U datasets
!pip install s3fs
import s3fs
from datasets import load_dataset
STORAGE_OPTIONS = {
"key": "...",
"secret": "...,
}
S3_BUCKET="s3://..."
fs = s3fs.S3FileSystem(**STORAGE_OPTIONS)
data = load_dataset("parquet", data_files=f"{S3_BUCKET}*.parquet", storage_options=STORAGE_OPTIONS)
```
---
**Author:**  [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [July 27, 2023, 7:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhelp-creating-dataset-from-s3-bucket-with-parquet-files%2F48429%2F2 "2023-07-27T19:01:12Z")
Support for `fsspec` filesystems is still experimental.
A quick test on a public S3 bucket shows there are still some issues to address to support S3 paths. I’m fixing them in [Fix `fsspec` download by mariosasko · Pull Request #6085 · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/pull/6085) (should be merged tomorrow). Then, it will be possible to download these files by installing `datasets` from `main`.
---
**Author:**  [@jjovalle99](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjjovalle99)%5C
**Post date:** [July 27, 2023, 7:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhelp-creating-dataset-from-s3-bucket-with-parquet-files%2F48429%2F3 "2023-07-27T19:34:02Z")
Awesome @mariosasko, I’ll be waiting for that then. Do you think my code setup is correct? Also, In case you have time, is any other approach that you consider will work fine in my scenario?