Using semantic segmentation

The task of image semantic segmentation consists of detecting specific regions of objects within an image.
So in image semantic segmentation, our goal is not only to identify the presence of given objects, but also to identify their positions, dimensions, and shape. This requires work at pixel level in order to classify every single pixel of the object. This can be achieved in an approximate way by drawing the shape of the object.

In Kili app, semantic segmentation can be:

• Standard semantic segmentation
• Interactive semantic segmentation

Standard semantic segmentation

To perform semantic segmentation:

1. Select a category.
2. Hover over a specific point of an image, and then press and hold your left mouse button.
3. While holding the left mouse button, draw the shape of the object.
4. To complete the shape, move your pointer to the your first point that you clicked on and release the left mouse button.

📘

If you release the left mouse button while drawing, the line you're drawing will change to a dashed line.
You can now complete your shape by making a diagonal connection to the first point that you clicked on.

Example image showing how a shape is completed using a diagonal, dashed line:

Example image with completed semantic segmentation on one object:

Interactive semantic segmentation

📘

In Image semantic segmentation projects, the interactive segmentation tool does not work with the following formats:

• .ico
• .webp
• .gif
• .tif/.tiff
• .jp2

The interactive segmentation tool only works with images smaller than 30 MB.

If the images are managed in a Cloud storage, Interactive segmentation will work only if the integration is set up with the Grant Kili authorization to process assets on its servers option activated.

To perform interactive semantic segmentation:

1. Select a category.
2. From the toolbar, select the interactive segmentation tool.
3. Click on the object that you want to classify. Object mask will be created automatically.

📘

• Interactive segmentation has been implemented with the SAM model working in the background, which means that the generated masks will be of very high quality. Getting the first interactively-generated mask may take a few seconds, but any subsequent masks will take much shorter to compute.
• You can interrupt the interactive segmentation process by pressing the ESC key.

4. Adjust the created mask:

• To add a specific region from the mask, click in the center of the region that you want to add to the mask. Region will be added automatically.
• To remove a specific region from the mask, press and hold Alt/Option and then click in the center of the region that you want to remove from the mask. Region will be removed automatically.
• To cancel and remove your mask, press the Escape key.

In our example, our category is: “Car without wheels”. We want to select the car and remove its wheels.

First, from the top menu, select the interactive segmentation tool, next, select the car. Finally, press and hold Alt/Option and then click in the center of the region that you want to remove from the mask:

📘

When using the Interactive Segmentation click tool (the magic wand icon), you can press the Enter key to validate the segmentation mask you’re working on and immediately start creating another one, without having to re-select the class and the tool with additional clicks. This speeds up the process of creating multiple annotations on the same image.

An alternative (and faster) way of adding annotations through interactive semantic segmentation is to use the Bounding box tool. When you use a bbox, you can more accurately target the object that you want to annotate.