# Custom dataset for MaskFormer and Mask2Former

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-maskformer-and-mask2former%2F112410%3E%5C
**Category:** Beginners\
**Created:** [October 16, 2024, 5:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-maskformer-and-mask2former%2F112410 "2024-10-16T17:05:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![olmobaldoni](https://avatars.discourse-cdn.com/v4/letter/o/7feea3/32.png) [@olmobaldoni](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Folmobaldoni)%5C
**Post date:** [October 16, 2024, 5:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-maskformer-and-mask2former%2F112410%2F1 "2024-10-16T17:05:42Z")

</div>

Hello everyone! I am a student currently trying to fine-tune the [MaskFormer](https://huggingface.co/docs/transformers/model_doc/maskformer) and [Mask2Former](https://huggingface.co/docs/transformers/main/model_doc/mask2former) segmentation model for an instance segmentation task on a custom dataset. The dataset I am preparing consists of two classes: `background` and `unhealty`. In each image of the dataset, there are several instances of the `unhealty` class. What I am trying to figure out is how to structure the dataset. I am not using a COCO format for the annotations. The tutorials I have found are mainly aimed at semantic segmentation tasks and I have not found much material on how to prepare custom datasets for these models. Thank you in advance for your help!

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/64626_2.png) [@mahmutc](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmahmutc)%5C
**Post date:** [October 17, 2024, 12:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-maskformer-and-mask2former%2F112410%2F2 "2024-10-17T12:10:34Z")

</div>

hi @olmobaldoni

I’m not very familiar with image stuff but this looks promising:

> <https://github.com/NielsRogge/Transformers-Tutorials/blob/master/MaskFormer/Fine-tuning/Fine_tune_MaskFormer_on_an_instance_segmentation_dataset_(ADE20k_full).ipynb>

mostly “Create PyTorch DataLoader” part.

Or, are you looking a tool to annotate your images?

---

<div class="post-metadata">

**Author:** ![olmobaldoni](https://avatars.discourse-cdn.com/v4/letter/o/7feea3/32.png) [@olmobaldoni](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Folmobaldoni)%5C
**Post date:** [October 17, 2024, 12:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-maskformer-and-mask2former%2F112410%2F3 "2024-10-17T12:52:03Z")

</div>

I’ve done some research on the notebook in question, but it seems to rely on a pre-existing dataset. I would like to clarify a couple of things:

1. Should the segmentation mask display all instances, or for a single image in my dataset, do I have a separate segmentation mask for each instance within that image?
2. How should the various masks be encoded? I’ve read elsewhere that an RGB annotation is used, where the red channel represents the class of the instance and the blue channel distinguishes different instances within the same class. However, I’m not entirely sure about this, and I’m curious about how the missing channel is utilized.

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/64626_2.png) [@mahmutc](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmahmutc)%5C
**Post date:** [October 17, 2024, 1:58pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-maskformer-and-mask2former%2F112410%2F4 "2024-10-17T13:58:23Z")

</div>

```auto
batch["class_labels"][batch_index]
tensor([ 1, 1, 2, 5, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 9, 10, 13, 13,
        17, 28, 36, 51, 51, 51, 51, 51, 51, 53, 87, 87, 90])

```

```auto
batch["mask_labels"][batch_index].shape
torch.Size([31, 512, 512])

```

According to notebook you need separate mask for each instance, right? The next masks/images in the notebook refer to two windowpane(class\_label = 1) instances from the same image(batch\_index = 1).

This one should get the cabinet (class\_label = 2) and the next one should be a table etc… :

```auto
print("Visualizing mask for:", id2label[batch["class_labels"][batch_index][2].item()])
visual_mask = (batch["mask_labels"][batch_index][2].bool().numpy() * 255).astype(np.uint8)
Image.fromarray(visual_mask)

```

Please read this part as well:  
`Next, we provide those to the image processor, which will turn the single instance segmentation map into a set of binary masks and corresponding labels. This is the format that MaskFormer expects (as it casts any image segmentation task to this format - also called "binary mask classification").`
