# Should I use BertConfig? Why these output are different?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-i-use-bertconfig-why-these-output-are-different%2F14579%3E%5C
**Category:** Beginners\
**Created:** [February 10, 2022, 12:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-i-use-bertconfig-why-these-output-are-different%2F14579 "2022-02-10T12:41:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![anon44965279](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@anon44965279](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon44965279)%5C
**Post date:** [February 10, 2022, 12:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-i-use-bertconfig-why-these-output-are-different%2F14579%2F1 "2022-02-10T12:41:55Z")

</div>

Why are these output different?

```auto
from transformers import AutoModel, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('prajjwal1/bert-tiny')
model = AutoModel.from_pretrained('prajjwal1/bert-tiny', output_hidden_states=True)
print(model(**tokenizer(sent, return_tensors="pt"), output_hidden_states=True).hidden_states)

```

```auto
from transformers import AutoModelForMaskedLM, AutoTokenizer

    tokenizer = AutoTokenizer.from_pretrained('prajjwal1/bert-tiny')
    model = AutoModelForMaskedLM.from_pretrained('prajjwal1/bert-tiny', output_hidden_states=True)
    print(model(**tokenizer(sent, return_tensors="pt"), output_hidden_states=True).hidden_states)

```

Actually, the first two outputs are exactly the same, but what about this third code?

```auto
 from transformers import AutoConfig, AutoModelForMaskedLM, AutoTokenizer
    tokenizer = AutoTokenizer.from_pretrained('prajjwal1/bert-tiny')
    config = AutoConfig.from_pretrained('prajjwal1/bert-tiny', output_hidden_states=True)
    model = AutoModelForMaskedLM.from_config(config)
    print(model(**tokenizer(sent, return_tensors="pt"), output_hidden_states=True).hidden_states)

```

---

<div class="post-metadata">

**Author:** ![beneyal](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/beneyal/32/27988_2.png) [@beneyal](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbeneyal)%5C
**Post date:** [February 11, 2022, 8:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-i-use-bertconfig-why-these-output-are-different%2F14579%2F2 "2022-02-11T08:57:51Z")

</div>

Hello 🙂

The 1st and 2nd snippets load the weights of the `prajjwal1/bert-tiny` model (either with or without the LM head), so their outputs are the same.

The 3rd snippet only loads the config, meaning no weights are loaded, the `model` variable contains an untrained model, so the outputs will differ.
