# How to apply DataCollator to a dataset for evaluation?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-apply-datacollator-to-a-dataset-for-evaluation%2F40864%3E%5C
**Category:** Beginners\
**Created:** [May 24, 2023, 5:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-apply-datacollator-to-a-dataset-for-evaluation%2F40864 "2023-05-24T17:56:01Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![henvoni](https://avatars.discourse-cdn.com/v4/letter/h/ad7895/32.png) [@henvoni](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fhenvoni)%5C
**Post date:** [May 24, 2023, 5:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-apply-datacollator-to-a-dataset-for-evaluation%2F40864%2F1 "2023-05-24T17:56:01Z")

</div>

i have a test dataset with the following structure:

```auto
tokenized_data
=> Dataset({
    features: ['id', 'tokens', 'ner_tags', 'input_ids', 'token_type_ids', 'attention_mask', 'labels'],
    num_rows: 2672
})

```

I am trying to use the Evaluator class like so:

```auto
task_evaluator = evaluate.evaluator('token-classification')
eval_results = task_evaluator.compute(model_or_pipeline=model,
                                      tokenizer=tokenizer,
                                      data=tokenized_data,
                                      metric='seqeval')

```

However, data is not truncated yet and therefore i get an error:

```auto
RuntimeError: The size of tensor a (951) must match the size of tensor b (512) at non-singleton dimension 1

```

Is there any way to pass a DataCollator object to the evaluator?
