BERT model trained on small corpus (English)?

macabdul9 May 8, 2021, 10:06pm 1

Is there any transformer-based model pre-trained on very small data ( less than BERT)?

Topic		Replies	Views
Further pre-train language model in transformers like BERT Models	3	1137	March 27, 2022
Using EXTREMELY small dataset to finetune BERT 🤗Transformers	6	13578	February 1, 2023
pre-train_BERT for a specific corpus 🤗Transformers	0	80	May 2, 2024
Small miniLM model for multilingual 🤗Transformers	0	336	October 7, 2021
Has anyone come across BERT fine-tuned for CLM task? 🤗Transformers	0	88	March 8, 2024