# Training PEGASUS on unlabeled data

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-pegasus-on-unlabeled-data%2F5565%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 16, 2021, 8:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-pegasus-on-unlabeled-data%2F5565 "2021-04-16T20:17:06Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Skylixia](https://avatars.discourse-cdn.com/v4/letter/s/439d5e/32.png) [@Skylixia](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSkylixia)%5C
**Post date:** [April 16, 2021, 8:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-pegasus-on-unlabeled-data%2F5565%2F1 "2021-04-16T20:17:06Z")

</div>

Hi,

I was wondering if there is any way to load a PEGASUS model and further train it on unlabeled data with its training objective rather than supervised finetuning with labeled data ?