# Fine tuning evaluation decode problem

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-evaluation-decode-problem%2F54045%3E%5C
**Category:** Intermediate\
**Created:** [September 8, 2023, 5:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-evaluation-decode-problem%2F54045 "2023-09-08T05:56:37Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![mHossain](https://avatars.discourse-cdn.com/v4/letter/m/4491bb/32.png) [@mHossain](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FmHossain)%5C
**Post date:** [September 8, 2023, 5:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-evaluation-decode-problem%2F54045%2F1 "2023-09-08T05:56:37Z")

</div>

Currently fine tuning a mT5 cross summarization model for english to bengali cross sum which trained on 47 language… In compute\_metric function its returns the prediction in english but supposed to be in bengali.

my code is here :

def compute\_metrics(eval\_pred):  
predictions, labels = eval\_pred  
decoded\_preds = tokenizer.batch\_decode(predictions,decoder\_start\_token\_id=get\_lang\_id(target\_lang), skip\_special\_tokens=True)  
# Replace -100 in the labels as we can’t decode them.  
labels = np.where(labels != -100, labels, tokenizer.pad\_token\_id)  
decoded\_labels = tokenizer.batch\_decode(labels,decoder\_start\_token\_id=get\_lang\_id(target\_lang), skip\_special\_tokens=True)

```
# Rouge expects a newline after each sentence
decoded_preds = ["\n".join(nltk.sent_tokenize(pred.strip())) for pred in decoded_preds]
decoded_labels = ["\n".join(nltk.sent_tokenize(label.strip())) for label in decoded_labels]

print(decoded_preds)
print(decoded_labels)
result = metric.compute(predictions=decoded_preds, references=decoded_labels, use_stemmer=True)
# Extract a few results
result = {key: value.mid.fmeasure * 100 for key, value in result.items()}

# Add mean generated length
prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in predictions]
result["gen_len"] = np.mean(prediction_lens)

return {k: round(v, 4) for k, v in result.items()}

```

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/8/3/8303dde056ac98f87d28eb6acb206bcf313fbcba.png)
