# Custom dataset maskformer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-maskformer%2F136079%3E%5C
**Category:** Intermediate\
**Created:** [January 16, 2025, 12:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-maskformer%2F136079 "2025-01-16T12:56:34Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![olmobaldoni](https://avatars.discourse-cdn.com/v4/letter/o/7feea3/32.png) [@olmobaldoni](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Folmobaldoni)%5C
**Post date:** [January 16, 2025, 7:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-maskformer%2F136079%2F6 "2025-01-16T19:43:29Z")

</div>

* * *

I decided to use `do_reduce_labels=True` together with `ignore_index=255`, following this discussion about a similar case:

> [@Understanding ignore\_index and reduce\_labels](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funderstanding-ignore-index-and-reduce-labels%2F64587%2F2)%3A
>
> Ignore\_index is the index that is ignored for loss calculations etc. set to 255 - as that’s used in standard loss functions I believe. Some models default to 255 and others don’t set it - not sure exactly why. This will take your background pixels and also unlabelled pixels and set them to the ignore\_index value Reduce\_labels - set to True. reindexes the label values so that they start from 0 after the background is set to 255

Additionally, I found a similar scenario in a tutorial for fine-tuning Mask2Former, where the `config.json` file also has `do_reduce_labels=True`. According to the documentation, the preprocessing for MaskFormer and Mask2Former should be identical.

> <https://github.com/huggingface/transformers/blob/94af1c0aa242d60faa8e69acf3391b1e4fd0d4bc/examples/pytorch/instance-segmentation/run_instance_segmentation.py#L403>

In the same file:

```python
# We need to specify the label2id mapping for the model
# it is a mapping from semantic class name to class index.
# In case your dataset does not provide it, you can create it manually:
# label2id = {"background": 0, "cat": 1, "dog": 2}
label2id = dataset["train"][0]["semantic_class_to_id"]

if args.do_reduce_labels:
    label2id = {name: idx for name, idx in label2id.items() if idx != 0} # remove background class
    label2id = {name: idx - 1 for name, idx in label2id.items()} # shift class indices by -1

```

From what I gather (though I’m not entirely sure, as the documentation isn’t very clear), when you don’t want to consider the background as a segmentable class, the preprocessor replaces the background in the image with the value `255`. This value is ignored during loss computation.

Thus, I set the parameter as follows. I don’t think the `ignore_index` value can be arbitrarily set (e.g., if I have `{0: 'garden', 1: 'car', 2: 'tree'}` and set `ignore_index=1`, the ‘car’ class will be ignored during loss computation).

The parameter `do_reduce_labels=True` ensures that classes start from 0 and increment upward, which is why they are shifted by `-1`.

* * *

Example (Models trained with 20 epochs and learning rate `5e-5`)

Test Image:

 ![image_1737056058631_0](https://us1.discourse-cdn.com/hellohellohello/original/3X/e/1/e131b06a432e947d09b604adba14a0464be50708.jpeg)

* * *

Preprocessor for MaskFormer:

```python
self.processor = AutoImageProcessor.from_pretrained(
    "facebook/maskformer-swin-small-coco",
    do_reduce_labels=True,
    reduce_labels=True,
    ignore_index=255,
    do_resize=False,
    do_rescale=False,
    do_normalize=False,
)

```

Results with MaskFormer:

```plaintext
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       Test metric DataLoader 0
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
        test_loss 1.0081120729446411
        test_map 0.038004860281944275
       test_map_50 0.06367719173431396
       test_map_75 0.040859635919332504
     test_map_large 0.5004204511642456
     test_map_medium 0.04175732284784317
     test_map_small 0.007470746990293264
       test_mar_1 0.01011560671031475
       test_mar_10 0.05838150158524513
      test_mar_100 0.06329479813575745
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

```

Test Image Result with MaskFormer:

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/2/2/22afdc697e1fa1a0fa40ea0985d62298696dc612.jpeg)

* * *

Preprocessor for Mask2Former:

```python
self.id2label = {0: "unhealty"}
self.label2id = {v: int(k) for k, v in self.id2label.items()}
self.processor = AutoImageProcessor.from_pretrained(
    "facebook/mask2former-swin-small-coco-instance",
    do_reduce_labels=True,
    reduce_labels=True,
    ignore_index=255,
    do_resize=False,
    do_rescale=False,
    do_normalize=False,
)

```

Results with Mask2Former:

```plaintext
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       Test metric DataLoader 0
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
        test_loss 15.374979972839355
        test_map 0.44928184151649475
       test_map_50 0.6224347949028015
       test_map_75 0.5011898279190063
     test_map_large 0.8390558958053589
     test_map_medium 0.6270320415496826
     test_map_small 0.32075226306915283
       test_mar_1 0.03526011481881142
       test_mar_10 0.24104046821594238
      test_mar_100 0.5274566411972046
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

```

Test Image Result with Mask2Former:

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/0/9/09b04f1850df069085120e83a43fb73f95e099af.jpeg)

* * *

As you can see, the results are very different, even though the code is identical, except for the parts where the model type is changed. if you want I can share the code.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-maskformer%2F136079)._
