# Questions about Checkpoints within HF

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-checkpoints-within-hf%2F6095%3E%5C
**Category:** Beginners\
**Created:** [May 10, 2021, 6:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-checkpoints-within-hf%2F6095 "2021-05-10T06:46:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![unknownTransformer](https://avatars.discourse-cdn.com/v4/letter/u/9f8e36/32.png) [@unknownTransformer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FunknownTransformer)%5C
**Post date:** [May 10, 2021, 6:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-checkpoints-within-hf%2F6095%2F1 "2021-05-10T06:46:46Z")

</div>

Hello Guys,

i have a few (prop) stupid questions about the HF-Checkpoint System.

1. While training i set `save_total_limit` to be 3 and i forghott to use `load_best_model_at_end`. Is the Last Checkpoint the best Result? Or do i have to validate all 3 Checkpoints on my own to find out which did best?

2. I have some problems with my Modell randomly crashing because i run out of CUDA-Memory (sometimes after 8 Hours, sometimes it runs without a problem). If i set `overwrite_output_dir` could i just do something like just running the same command a few times after each other to make sure the training finishes?

```auto
! python fine_tune.py [.. args ..]
! gsutil -> save results 
! python fine_tune.py [.. args ..]
! gsutil -> save results 
! python fine_tune.py [.. args ..]
! gsutil -> save results 

```

Ty in advcanded