# Accelerator .prepare() replaces custom DataLoader Sampler

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392%3E%5C
**Category:** 🤗Accelerate\
**Created:** [June 15, 2023, 2:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392 "2023-06-15T14:53:44Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![HTess](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/htess/32/16872_2.png) [@HTess](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHTess)%5C
**Post date:** [June 15, 2023, 2:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392%2F1 "2023-06-15T14:53:44Z")

</div>

I want to use a DataLoader that uses a custom sampler you can find at [vision/references/classification/sampler.py at main · pytorch/vision · GitHub](https://github.com/pytorch/vision/blob/main/references/classification/sampler.py)

When doing :

```auto
print(dataset, dataset.sampler)
dataset = accelerator.prepare(dataset)
print(dataset, dataset.sampler)

```

I get the following print:

```auto
<torch.utils.data.dataloader.DataLoader object at 0x7f987fbc4e50> <utils.imagenet_dataloader.RASampler object at 0x7f987fbc4e20>
<accelerate.data_loader.DataLoaderShard object at 0x7f98518b6da0> <torch.utils.data.sampler.SequentialSampler object at 0x7f98518b6b60>

```

This means that my RASampler got turned into a SequentialSampler.  
Is this a normal behaviour? Since it seems I can’t manually restore my sampler afterwhile, this is quite a problem.  
Could you tell me how to solve this problem?

---

<div class="post-metadata">

**Author:** ![varadhbhatnagar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/varadhbhatnagar/32/31383_2.png) [@varadhbhatnagar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvaradhbhatnagar)%5C
**Post date:** [February 3, 2024, 3:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392%2F2 "2024-02-03T15:53:24Z")

</div>

Hi @HTess  
Did you find a solution to this?

---

<div class="post-metadata">

**Author:** ![muellerzr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/muellerzr/32/21583_2.png) [@muellerzr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmuellerzr)%5C
**Post date:** [February 3, 2024, 4:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392%2F3 "2024-02-03T16:48:58Z")

</div>

> [@HTess](#):
>
> Is this a normal behaviour? Since it seems I can’t manually restore my sampler afterwhile, this is quite a problem.

Yes. The original sampler is still there I think as well? Can you check `sampler.sampler`? Or `sampler.batch_sampler`

---

<div class="post-metadata">

**Author:** ![varadhbhatnagar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/varadhbhatnagar/32/31383_2.png) [@varadhbhatnagar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvaradhbhatnagar)%5C
**Post date:** [February 3, 2024, 5:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392%2F4 "2024-02-03T17:13:48Z")

</div>

Hi @muellerzr

I wanted to use a custom sampler with my dataloader. Will the sampler behaviour remain the same after passing through `accelerate.prepare()` ? Or will it be changed to a `SequentialSampler()` ?

---

<div class="post-metadata">

**Author:** ![muellerzr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/muellerzr/32/21583_2.png) [@muellerzr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmuellerzr)%5C
**Post date:** [February 3, 2024, 5:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392%2F5 "2024-02-03T17:16:18Z")

</div>

As mentioned, the custom sampler will be used. This new sampler is simply just distributing all of the batch across your GPUs. So it goes old sampler → new sampler to dispatch.

To test this, you can try including a `print` statement in your custom sampler and iterate after `prepare`

---

<div class="post-metadata">

**Author:** ![razvan-a-matisan](https://avatars.discourse-cdn.com/v4/letter/r/df705f/32.png) [@razvan-a-matisan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frazvan-a-matisan)%5C
**Post date:** [March 9, 2025, 3:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392%2F6 "2025-03-09T15:03:32Z")

</div>

Hi! Is there a way to get the state of the custom sampler after .prepare() method was used?  
I would like to save the state of the custom sampler, but I am not sure if it’s possible?
