# Pretrain RoBERTa-large from scratch in Swedish

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-large-from-scratch-in-swedish%2F7497%3E%5C
**Category:** Flax/JAX Projects\
**Created:** [June 28, 2021, 9:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-large-from-scratch-in-swedish%2F7497 "2021-06-28T09:34:15Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![saattrupdan](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saattrupdan/32/28869_2.png) [@saattrupdan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsaattrupdan)%5C
**Post date:** [June 28, 2021, 5:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-large-from-scratch-in-swedish%2F7497%2F2 "2021-06-28T17:38:01Z")

</div>

Just a suggestion: How about we pretrain a common Scandinavian model instead, as our languages are so similar? I’ve made a project suggestion here: [Scandinavian RoBERTa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fscandinavian-roberta%2F7533).

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-large-from-scratch-in-swedish%2F7497)._
