# Disabling addition of CLS from BERT tokenizer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdisabling-addition-of-cls-from-bert-tokenizer%2F15538%3E%5C
**Category:** 🤗Tokenizers\
**Created:** [March 9, 2022, 2:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdisabling-addition-of-cls-from-bert-tokenizer%2F15538 "2022-03-09T14:41:44Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![AfonsoSousa](https://avatars.discourse-cdn.com/v4/letter/a/5f9b8f/32.png) [@AfonsoSousa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAfonsoSousa)%5C
**Post date:** [March 9, 2022, 2:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdisabling-addition-of-cls-from-bert-tokenizer%2F15538%2F1 "2022-03-09T14:41:44Z")

</div>

Hi. I am using the BERT Huggingface tokenizer for a sequence-to-sequence task. I am using an LSTM-based Encoder-Decoder architecture. I want my input to the decoder to start with the sep\_token followed by the target sentence shifted one character to the right. I built the sentence as I wished to, but when I tokenized it, the [CLS] token is always added to the beginning of the sentence. How can I disable this addition? Thanks in advance for any help you can provide.

---

<div class="post-metadata">

**Author:** ![christopher](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/christopher/32/31595_2.png) [@christopher](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchristopher)%5C
**Post date:** [March 11, 2022, 7:13am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdisabling-addition-of-cls-from-bert-tokenizer%2F15538%2F2 "2022-03-11T07:13:24Z")

</div>

Hello there.

The [CLS] token is added by the tokenizer’s post\_processor. I believe you will have to use the HF tokenizers library to define your own. See: [Input sequences — tokenizers documentation](https://huggingface.co/docs/tokenizers/python/latest/api/reference.html#module-tokenizers.processors)

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [March 11, 2022, 7:43am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdisabling-addition-of-cls-from-bert-tokenizer%2F15538%2F3 "2022-03-11T07:43:59Z")

</div>

You should be able to add `add_special_tokens=False` to the `tokenizer(your_input, add_special_tokens=False)` call.

---

<div class="post-metadata">

**Author:** ![christopher](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/christopher/32/31595_2.png) [@christopher](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchristopher)%5C
**Post date:** [March 11, 2022, 8:19am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdisabling-addition-of-cls-from-bert-tokenizer%2F15538%2F4 "2022-03-11T08:19:25Z")

</div>

Wouldn’t that also disable the sep\_token OP mentioned?

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [March 11, 2022, 8:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdisabling-addition-of-cls-from-bert-tokenizer%2F15538%2F5 "2022-03-11T08:39:18Z")

</div>

It seems to me that they want to add a SEP token at the start of the string, but that the other parts are already formatted as they want/do not need other special tokens. So they should be able to do something like this (untested):

```auto
encoded = tokenizer(your_input, add_special_tokens=False)
encoded["input_ids"] = [tokenizer.sep_token_id] + encoded["input_ids"]

```

---

<div class="post-metadata">

**Author:** ![AfonsoSousa](https://avatars.discourse-cdn.com/v4/letter/a/5f9b8f/32.png) [@AfonsoSousa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAfonsoSousa)%5C
**Post date:** [March 11, 2022, 9:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdisabling-addition-of-cls-from-bert-tokenizer%2F15538%2F6 "2022-03-11T09:39:42Z")

</div>

Thank you. That solved the issue.
