# Problems with understanding instruction fine-tuning **URL:** **Author:** ![vdw](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vdw/32/22539_2.png) [@vdw](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvdw)%5C **Post date:** [April 2, 2024, 9:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fproblems-with-understanding-instruction-fine-tuning%2F79799%2F1 "2024-04-02T09:26:56Z") I’m trying to read up on instruction fine-tuning, but I think I have a big misunderstanding. As I understand, instruction datasets typically have 3 components: (a) the instruction (b) the output/response, and (c) and an optional input. Now, according to this [paper](https://arxiv.org/pdf/2308.10792.pdf): “Based on the collected IT dataset, a pretrained model can be directly fine-tuned in a fully-supervised manner, where given the instruction and the input, the model is trained by predicting each token in the output sequentially.” This makes sense to me, i.e, the response/output is the ground truth the model is expected to predict. However, when I check many tutorials (e.g., this [tutorial notebook](https://github.com/philschmid/deep-learning-pytorch-huggingface/blob/main/training/instruction-tune-llama-2-int4.ipynb)), it seems that instructions, inputs, and outputs are all combined into a single text sample. But I can’t tell from the notebook how the training now works. What is now the ground truth for the supervised training. Or is this now treated as a next-word-prediction task? What am I missing? Or are these indeed two different approaches for instruction tuning. Sorry if those a stupid questions!