DocumentationRecipesReferenceGraphQLChangelog
Log In

Handling questions and issues

As part of the QA flow, you can create issues or address the questions asked by labelers. When all the issues and questions are addressed, you can resolve the issue.

Creating issues

When you are reviewing labeled assets, you can add issues:

Adding an issue to an asset

To add an issue to an asset:

  1. Click on the Issues & Questions button.
Issues & Questions button

Issues & Questions button

  1. Type your comment to explain the issue and then click on Add issue.

Adding an issue to a specific annotation

πŸ“˜

Currently, you can add issues to these types of annotations:

  • Bounding boxes
  • Polygons
  • Named entities
  1. From the asset viewer, right-click an annotation.
  2. From the context menu, select Add issue.
  1. Type your comment to explain the issue and then click on Add issue.

When you're done adding issues, click on Send back. The asset will re-appear in labeler's queue and the status of the asset will change from Labeled to Ongoing.

πŸ“˜

If an asset is sent back to queue, it gets top labeling priority. This means that the labeler will see this asset as the first one to be labeled.


Addressing the questions asked by labelers

πŸ“˜

You can filter displayed assets to show only those that have open questions/issues linked to them. You can do it from the project Queue page or from the Explore view, using the advanced query builder.

Answering questions

Reviewers and admins can answer questions in the review. To answer a question:

  1. Click on the Issues & Questions button to access open questions.
Issues & Questions button

Issues & Questions button

  1. Type your answer and then click on Reply.
  1. When you're done answering questions, click on Send back. The asset will re-appear in labeler's queue and the status of the asset will change from Labeled to Ongoing.

πŸ“˜

  • If a labeler added a question that is related to a specific label, reviewers see the question icon next to the label as well.
  • If an asset is sent back to queue, it gets top labeling priority. This means that the labeler will see this asset as the first one to be labeled.

Resolving issues

When the labeler corrects the annotation and re-submits the asset, reviewers can resolve the issue by either of these actions: