Queue prioritization

Kili app manages a queue for each project. The queue can be customized and redefined throughout the project. This is helpful for advanced annotation processes for business expectations, priorities, active learning, production supervision, and real-time annotation.

Three strategies are possible:

• First In First Out (FiFo)
• Assigning assets to specific labelers
• Prioritizing assets

First In First Out (FiFo)

FiFo is the default queue orchestration. Assets are loaded based on their time stamp (creation date): assets uploaded as the first ones will be distributed to be annotated first.

Assigning assets to specific labelers

You can customize the queue to assign a specific list of assets to specific labelers.
The assignments can work for the whole queue or for a specific subset of the dataset.

📘

Assigning assets does not impact their usual order of appearance in the labeling queue. It just ensures that some assets are specifically labeled by one (or several) labelers; for example, because of their unique ability to process such assets. For information on how to prioritize assets, refer to Prioritizing assets.

Assigning assets to labelers from the Kili app interface

To assign assets to labelers from the Kili app interface:

1. From the Queue page, select the assets.
2. From the quick actions menu, select Assign to.
3. Type the name of the labeler, and then click Assign.

Assigning assets to labelers programmatically

Refer to Kili API documentation:

• GraphQL API key: toBeLabeledBy.
• Python SDK, function: update_properties_in_assets, argument: to_be_labeled_by_array.

For examples, refer to this recipe:

Prioritizing assets

You can set specific labeling priorities at the asset level. The higher the number, the higher the priority. If two assets have the same priority, FiFo applies.

For example:

• If asset 1 has priority N_1 and asset 2 has priority N_2, and N_1 > N_2, then asset 1 has higher priority than asset 2 and will be distributed first.
• If N_1 = N_2, priority follows the default strategy (FiFo).

Prioritizing assets from the Kili app interface

To prioritize assets from the the Kili app interface:

1. From the Queue page, select the assets.
2. From the quick actions menu, select Prioritize.
3. Select one of the predefined options or type the priority manually, and then click Apply.

Prioritizing assets programmatically

You can prioritize assets with the Kili SDK, using the priorities parameter of the update_properties_in_assets method (for details, refer to the documentation).

For an end-to-end example of how to programmatically prioritize your project queue and assign specific assets to specific labelers using Kili's Python SDK, refer to our tutorial on how to set up workflows.

Additionally, here’s a dedicated recipe that helps you assign asset priorities:

📘

You can also prioritize assets based on Kili AutoML recommendations.