# Continue from pretrained

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcontinue-from-pretrained%2F40472%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 20, 2023, 3:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcontinue-from-pretrained%2F40472 "2023-05-20T15:14:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![hamedhm](https://avatars.discourse-cdn.com/v4/letter/h/7ab992/32.png) [@hamedhm](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhamedhm)%5C
**Post date:** [May 20, 2023, 3:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcontinue-from-pretrained%2F40472%2F1 "2023-05-20T15:14:09Z")

</div>

I am training a transformer model and since it is huge, I need to stop training after each epoch and continue. I am doing the following. Please can you check and confirm that this indeed continues training from the previous epoch:

```auto
from transformers import RobertaForMaskedLM
model = RobertaForMaskedLM.from_pretrained('path to the previous epoch pretrained')
model.train()
...

```

---

<div class="post-metadata">

**Author:** ![ianuragbhatt](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ianuragbhatt/32/16236_2.png) [@ianuragbhatt](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fianuragbhatt)%5C
**Post date:** [May 21, 2023, 10:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcontinue-from-pretrained%2F40472%2F2 "2023-05-21T10:44:20Z")

</div>

I think it’s right we load our saved model like that and then retraining from there.