# Save map cache to s3 bucket
**URL:**
**Author:**  [@Ollie](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FOllie)%5C
**Post date:** [September 9, 2021, 7:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsave-map-cache-to-s3-bucket%2F9838%2F1 "2021-09-09T07:03:31Z")
I’m currently making use of gcsfuse to load data from arrow tables stored in a Google Cloud Storage bucket mounted to a vm folder. The tables were saved automatically when the map() function cached the datasets to the mounted folder which is all well and good but gcsfuse often struggles with large datasets (io errors, sluggish, etc.).
Is there a way of using the equivalent of save\_to\_disk for the map() function’s caching? That way I could use the s3 filesystem to load the tables in a more stable fashion.
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [September 9, 2021, 4:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsave-map-cache-to-s3-bucket%2F9838%2F2 "2021-09-09T16:40:13Z")
Hi ! When do you experience slow downs or error exactly ?
Have you tried making `map` write to a local directory first (you can pass `cache_file_name` to `map` to specify where to write the Arrow data), and then save it to your cloud storage using `save_to_disk` ?
This way accessing the data from your processed dataset will be much faster since your dataset will be on your disk.
---
**Author:**  [@Ollie](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FOllie)%5C
**Post date:** [September 9, 2021, 5:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsave-map-cache-to-s3-bucket%2F9838%2F3 "2021-09-09T17:14:11Z")
Thanks for the reply! I should have mentioned that the VM has 100GB of disk space in total and the dataset in question is larger than this. By a factor of 5 or so. There is the option of making a new one but I’d like that to be a last resort if possible.
However, now that you mention it, there’s no reason why I can’t just map smaller batches of the data frame it originates from and `save_to_disk` each local table before deleting it - then using `concatenate_datasets` afterwards. Very messy though, seeing as each mapping process already uses 30 workers (along with `concatenate_datasets`).
The error I get for writing to the mounted folder is along the lines of `IOError: [Errno 5] Input/output error` which is also common when writing large files to a mounted Google Drive folder when using Colab. The slow-downs refer to timeouts when reading the arrow files. The kernel won’t stop but I need to run some of the cells again which means having to continuously monitor the process (which takes a long time).
Thanks!