# Early stopping implementation in accelerate?
**URL:**
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [September 7, 2022, 6:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fearly-stopping-implementation-in-accelerate%2F22694%2F1 "2022-09-07T18:15:31Z")
Is it possible to have an implementation of early stopping while using Accelerate? I know accelerate handles distributed training for normal pytorch training loops, but I’m not quite sure how to handle early stopping since one process could meet the early stop criteria and another may not. I was thinking of something like this:
```python
for epoch in range(num_epochs):
for batch in train_dataloader:
optimizer.zero_grad()
outputs = my_model(**batch)
loss = outputs['loss']
my_accelerator.backward(loss)
optimizer.step()
metric = my_eval(my_model, dev_dataloader) # evalution on dev set (i.e., holdout from training)
if my_early_stop.step(metric):
break # early stop criterion is met, we can stop now
```
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [September 7, 2022, 6:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fearly-stopping-implementation-in-accelerate%2F22694%2F2 "2022-09-07T18:36:07Z")
Why would the process see different metrics? They’ll all have the same one normally.
---
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [September 7, 2022, 6:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fearly-stopping-implementation-in-accelerate%2F22694%2F3 "2022-09-07T18:40:31Z")
@sgugger Thanks for your response. Maybe my understanding of accelerate is incorrect, but I thought each process saw different slices of the training and dev sets. Each process would compute the same metric but on different slices of the datasets.
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [September 7, 2022, 6:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fearly-stopping-implementation-in-accelerate%2F22694%2F4 "2022-09-07T18:45:09Z")
In which case you should gather the tensors before feeding them to your metric function, as is done in all examples.
---
**Author:**  [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [September 7, 2022, 6:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fearly-stopping-implementation-in-accelerate%2F22694%2F5 "2022-09-07T18:47:25Z")
Ah yes, thank you! Apologies for the simple question. I’m still learning Accelerate. Thank you for your help!