# Struggling in Cross lingual Summarization by mt5-base

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fstruggling-in-cross-lingual-summarization-by-mt5-base%2F58202%3E%5C
**Category:** Intermediate\
**Created:** [October 11, 2023, 5:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fstruggling-in-cross-lingual-summarization-by-mt5-base%2F58202 "2023-10-11T17:25:14Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![mHossain](https://avatars.discourse-cdn.com/v4/letter/m/4491bb/32.png) [@mHossain](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FmHossain)%5C
**Post date:** [October 11, 2023, 5:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fstruggling-in-cross-lingual-summarization-by-mt5-base%2F58202%2F1 "2023-10-11T17:25:14Z")

</div>

I am trying to fine tune mt5-base model to get summary in bengali from english sentence. but while decoding model always gives result in english not in bengali. how can i control or can i give instruction to target language detection.

here is some of my codes:

model\_checkpoint1 = “google/mt5-small”

tokenizer = AutoTokenizer.from\_pretrained(model\_checkpoint1, use\_fast=False)  
mt5\_config = AutoConfig.from\_pretrained(model\_checkpoint1)  
mt5\_config.decoder\_start\_token\_id = 250042  
model = AutoModelForSeq2SeqLM.from\_pretrained(model\_checkpoint1,config=mt5\_config)

custom\_tokenizer\_config = {  
“additional\_special\_tokens”: [“▁\<extra\_id\_69\>”, “▁\<extra\_id\_57\>”, “▁\<extra\_id\_58\>”, “▁\<extra\_id\_0\>”],  
}

tokenizer.add\_special\_tokens(custom\_tokenizer\_config)

# data preprocess

max\_input\_length = 1024  
max\_target\_length = 128  
prefix = “en to bn:”  
prefix1 = “▁\<extra\_id\_57\>”  
def preprocess\_function(examples):  
inputs = [prefix + doc for doc in examples[“text”]]

```
# Ensure the prefix is added to the "summary" field
summaries = [prefix1 + summary for summary in examples["summary"]]
print(summaries)

model_inputs = tokenizer(inputs, max_length=max_input_length, truncation=True)

# Setup the tokenizer for targets
with tokenizer.as_target_tokenizer():
    labels = tokenizer(summaries, max_length=max_target_length, truncation=True)
print(labels)
model_inputs["labels"] = labels["input_ids"]
return model_inputs

```

#compute metrics

import nltk  
import numpy as np  
nltk.download(‘punkt’)

def compute\_metrics(eval\_pred):  
predictions, labels = eval\_pred  
decoded\_preds = tokenizer.batch\_decode(predictions, skip\_special\_tokens=True)  
# Replace -100 in the labels as we can’t decode them.  
labels = np.where(labels != -100, labels, tokenizer.pad\_token\_id)  
decoded\_labels = tokenizer.batch\_decode(labels, skip\_special\_tokens=True)

```
# Rouge expects a newline after each sentence
decoded_preds = ["\n".join(nltk.sent_tokenize(pred.strip())) for pred in decoded_preds]
decoded_labels = ["\n".join(nltk.sent_tokenize(label.strip())) for label in decoded_labels]

print(decoded_preds)
print(decoded_labels)
result = metric.compute(predictions=decoded_preds, references=decoded_labels, use_stemmer=True)
# Extract a few results
result = {key: value.mid.fmeasure * 100 for key, value in result.items()}

# Add mean generated length
prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in predictions]
result["gen_len"] = np.mean(prediction_lens)

return {k: round(v, 4) for k, v in result.items()}

```

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/7/0/7021efbf9751e4e05a63f603a67373b25ea4d2a4.png)
