# Tutorial notebooks

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 10, 2020, 3:31pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658 "2020-08-10T15:31:13Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [August 10, 2020, 3:31pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F1 "2020-08-10T15:31:13Z")

</div>

Notebooks are now automatically created from the tutorials in the documentation of transformers. you can find them all [here](https://github.com/huggingface/notebooks/tree/master/transformers_doc) or click on the brand new “Open in colab” button on the doc pages that have one.

Plan is to add more tutorials and the corresponding notebooks very soon. I’ll focus on integration with nlp datasets and the Trainer in the next few weeks, but if you have ideas of tutorials you’d like to see, please let me know here.

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 10, 2020, 3:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F2 "2020-08-10T15:46:54Z")

</div>

Awesome! Thanks @sgugger

---

<div class="post-metadata">

**Author:** ![mrm8488](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mrm8488/32/5836_2.png) [@mrm8488](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmrm8488)%5C
**Post date:** [August 10, 2020, 4:02pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F3 "2020-08-10T16:02:05Z")

</div>

I would like to see a tutorial about integrating nlp with train a LM from scratch

---

<div class="post-metadata">

**Author:** ![rbint](https://avatars.discourse-cdn.com/v4/letter/r/85e7bf/32.png) [@rbint](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frbint)%5C
**Post date:** [August 11, 2020, 5:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F4 "2020-08-11T17:53:31Z")

</div>

Very excited for nlp datasets and the Trainer as I’ve had difficulty implementing them after the recent changes to load\_dataset()

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [August 11, 2020, 10:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F5 "2020-08-11T22:20:18Z")

</div>

If I’m correct, the examples are planned to be converted so that they will use the nlp datasets package. @mrm8488 @rbin

---

<div class="post-metadata">

**Author:** ![mrm8488](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mrm8488/32/5836_2.png) [@mrm8488](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmrm8488)%5C
**Post date:** [August 12, 2020, 1:27am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F6 "2020-08-12T01:27:40Z")

</div>

Yes, I am waiting for it!

---

<div class="post-metadata">

**Author:** ![aclifton314](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aclifton314/32/55465_2.png) [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [August 12, 2020, 8:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F7 "2020-08-12T20:16:16Z")

</div>

@mrm8488, this may not be exactly what you’re looking for but I have been finetuning GPT2 with a language model head using my own loss function. I tried to detail in [this post](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-gpt2-with-user-defined-loss%2F163%2F30) as much of the implementation as possible. Maybe parts of it will be useful for you as well since I am also using `Trainer`.

---

<div class="post-metadata">

**Author:** ![mrm8488](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mrm8488/32/5836_2.png) [@mrm8488](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmrm8488)%5C
**Post date:** [August 13, 2020, 7:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F8 "2020-08-13T07:46:24Z")

</div>

Thank you so much! I will give a try!

---

<div class="post-metadata">

**Author:** ![peggy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/peggy/32/286_2.png) [@peggy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpeggy)%5C
**Post date:** [August 24, 2020, 3:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F9 "2020-08-24T15:08:01Z")

</div>

Hi, I think an example of multi-task learning would be great. Especially one with different loss functions for each task.

---

<div class="post-metadata">

**Author:** ![saprativa](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saprativa/32/2659_2.png) [@saprativa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsaprativa)%5C
**Post date:** [February 14, 2022, 7:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftutorial-notebooks%2F658%2F10 "2022-02-14T19:27:45Z")

</div>

do you have any multi-task learning code for reference?
