Using semantic segmentation

The task of image semantic segmentation consists of detecting specific regions of objects within an image.
So in image semantic segmentation, our goal is not only to identify the presence of given objects, but also to identify their positions, dimensions, and shape. This requires work at pixel level in order to classify every single pixel of the object. This can be achieved in an approximate way by drawing the shape of the object.

In Kili app, semantic segmentation can be:

• Standard semantic segmentation
• Interactive semantic segmentation

Standard semantic segmentation

To perform semantic segmentation:

1. Select a category.
2. Hover over a specific point of an image, and then press and hold your left mouse button.
3. While holding the left mouse button, draw the shape of the object.
4. To complete the shape, move your pointer to the your first point that you clicked on and release the left mouse button.

📘

If you release the left mouse button while drawing, the line you're drawing will change to a dashed line.
You can now complete your shape by making a diagonal connection to the first point that you clicked on.

Example image showing how a shape is completed using a diagonal, dashed line:

Example image with completed semantic segmentation on one object:

Interactive semantic segmentation

📘

In Image semantic segmentation projects, the interactive segmentation tool does not work with the following formats:

• .ico
• .webp
• .gif
• .tif/.tiff

The interactive segmentation tool only works with images smaller than 30 MB.

To perform interactive semantic segmentation:

1. Select a category.
2. From the toolbar, select the interactive segmentation tool.
3. Click on the object that you want to classify. Object mask will be created automatically.

📘

Interactive segmentation has been implemented with the SAM model working in the background, which means that the generated masks will be of very high quality. Getting the first interactively-generated mask may take a few seconds, but any subsequent masks will take much shorter to compute.

4. Adjust the created mask:

• To add a specific region from the mask, click in the center of the region that you want to add to the mask. Region will be added automatically.
• To remove a specific region from the mask, press and hold Alt/Option and then click in the center of the region that you want to remove from the mask. Region will be removed automatically.
• To cancel and remove your mask, press the Escape key.

In our example, our category is: “Car without wheels”. We want to select the car and remove its wheels.

First, from the top menu, select the interactive segmentation tool, next, select the car. Finally, press and hold Alt/Option and then click in the center of the region that you want to remove from the mask: