Reviewing labeled assets

You can review assets in two ways:

• You manually select the annotated assets.
• Assets are randomly selected for review from the review queue.

In both cases, the review interface is similar to the labeling interface and allows you to review labeled assets and then:

• Make corrections, if necessary
• Send assets back to the queue

Manually selecting assets to be reviewed

To manually select labeled assets for review:

1. From your project Queue page, run Filters to search for labeled assets
2. From the Quick actions menu, select Add to review

Reviewing labeled assets

1. From your project Queue page, or from the Explore view, run Filters to search for assets labeled To review.
2. Click to open a specific asset that you want to review.
3. If the labels were added correctly, click Review.

🚧

Be careful when reviewing assets that are used as honeypot. Clicking the Review button changes the current honeypot setting for the asset.

4. If the labels were added incorrectly, you can make corrections, if necessary or send assets back to labeling.

Sending assets back to labeling

To send an asset back to the labeler:

1. From your project Queue page, run Filters to search for specific assets
2. Select assets
3. From the Quick actions menu, click Send back to queue

Alternatively, you can use the issue button:

1. From your project Queue page, or from the Explore view run Filters to search for specific assets.
2. Open the asset.
3. Add an issue. Refer to Handling questions and issues.

Selected assets will return to the project Queue page with status Ongoing.

📘

• The asset returns to the queue of the same labeler who added the disputed label. Kili does not impose any specific timeframe for the labeler to update the label.
• If an asset is sent back to queue, it gets top labeling priority. This means that the labeler will see this asset as the first one to be labeled.