# How to use models in tranformers?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-models-in-tranformers%2F23890%3E%5C
**Category:** Beginners\
**Created:** [October 2, 2022, 11:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-models-in-tranformers%2F23890 "2022-10-02T11:15:20Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![selfisheagle](https://avatars.discourse-cdn.com/v4/letter/s/8dc957/32.png) [@selfisheagle](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fselfisheagle)%5C
**Post date:** [October 2, 2022, 11:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-models-in-tranformers%2F23890%2F1 "2022-10-02T11:15:20Z")

</div>

I am very new to using transformers. I have just started using them, and I require them for a classification task, Where I have tweets from topics, and I need to find their hate score. I have been suggested to use the [GroNLP/hateBERT · Hugging Face](https://huggingface.co/GroNLP/hateBERT) model to do so.

But, I was trying it out by the following code:

```auto
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("GroNLP/hateBERT")

model = AutoModelForMaskedLM.from_pretrained("GroNLP/hateBERT")

```

Then I used the following code:

```auto
from transformers import pipeline

classifier = pipeline('text-classification', model=model, tokenizer=tokenizer)

```

But I am getting an error:  
`The model 'TFBertLMHeadModel' is not supported for text-classification. Supported models are ['TFAlbertForSequenceClassification', 'TFBertForSequenceClassification', 'TFCamembertForSequenceClassification', 'TFConvBertForSequenceClassification', 'TFCTRLForSequenceClassification', 'TFDebertaForSequenceClassification', 'TFDebertaV2ForSequenceClassification', 'TFDistilBertForSequenceClassification', 'TFElectraForSequenceClassification', 'TFFlaubertForSequenceClassification', 'TFFunnelForSequenceClassification', 'TFGPT2ForSequenceClassification', 'TFGPTJForSequenceClassification', 'TFLayoutLMForSequenceClassification', 'TFLayoutLMv3ForSequenceClassification', 'TFLongformerForSequenceClassification', 'TFMobileBertForSequenceClassification', 'TFMPNetForSequenceClassification', 'TFOpenAIGPTForSequenceClassification', 'TFRemBertForSequenceClassification', 'TFRobertaForSequenceClassification', 'TFRoFormerForSequenceClassification', 'TFTapasForSequenceClassification', 'TFTransfoXLForSequenceClassification', 'TFXLMForSequenceClassification', 'TFXLMRobertaForSequenceClassification', 'TFXLNetForSequenceClassification'].`

Can someone suggest how to modify this code so I can use the hateBERT model to find the hateful score? And any resources to learn about using any generic models would be very helpful.

---

<div class="post-metadata">

**Author:** ![stevhliu](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/stevhliu/32/18373_2.png) [@stevhliu](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstevhliu)%5C
**Post date:** [October 3, 2022, 4:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-models-in-tranformers%2F23890%2F2 "2022-10-03T16:34:29Z")

</div>

Hello and welcome @selfisheagle! 🤗

You’re getting an error because you loaded a model with `AutoModelForMaskedLM` instead of `AutoModelForSequenceClassification`, which is what you want for your classification task:

```nohighlight
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

tokenizer = AutoTokenizer.from_pretrained("GroNLP/hateBERT")
model = AutoModelForSequenceClassification.from_pretrained("GroNLP/hateBERT")
classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)

```

For more resources, check out the [docs](https://huggingface.co/docs/transformers/autoclass_tutorial#automodel) about how to use `AutoModel` to load a model for a task or [Chapter 2](https://huggingface.co/course/chapter2/1?fw=pt) of the Hugging Face course.

---

<div class="post-metadata">

**Author:** ![selfisheagle](https://avatars.discourse-cdn.com/v4/letter/s/8dc957/32.png) [@selfisheagle](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fselfisheagle)%5C
**Post date:** [October 3, 2022, 4:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-models-in-tranformers%2F23890%2F3 "2022-10-03T16:55:17Z")

</div>

Thanks for your response.  
When I ran the code you gave by replacing the sentiment-analysis part with text classification, I thought it would give hate vs no hate classification. But instead, I get the response.

> Some weights of the model checkpoint at GroNLP/hateBERT were not used when initializing BertForSequenceClassification: [‘cls.predictions.decoder.weight’, ‘cls.predictions.transform.LayerNorm.bias’, ‘cls.predictions.transform.LayerNorm.weight’, ‘cls.predictions.transform.dense.bias’, ‘cls.predictions.decoder.bias’, ‘cls.predictions.transform.dense.weight’, ‘cls.predictions.bias’]
> 
> - This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
> - This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).  
> Some weights of BertForSequenceClassification were not initialized from the model checkpoint at GroNLP/hateBERT and are newly initialized: [‘classifier.weight’, ‘classifier.bias’]  
> You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.  
> So, do I have to train still the model for hate vs no hate classification, I thought hateBERT would already have done something along those lines, and I don’t have to pre-train the model.
