DocumentationRecipesReferenceGraphQLChangelog
Log In

Getting started with Kili

This topic will guide you through a typical Kili project lifecycle. You will learn how to:

  1. Set up a project
  2. Add assets to your project
  3. Build project ontology using tools made available by Kili
  4. Add users to project and assign their roles in the project
  5. Properly configure quality management metrics
  6. Review labeled assets
  7. Analyze project KPIs
  8. Export annotations

Setting up a project

A project is a space where the work on data is done. This is where you configure the labeling jobs, import assets, supervise the production of labels and export the completed work.

Projects are created inside an organization. Only invited users can contribute.

Regarding your workflow and operational constraints, you can adopt two different approaches to labeling in Kili:

  • One-off project: Upload data > label > export labels.
  • Continuous work: When your initial labeling is done, you still regularly add new assets to label and improve your model.

You can either create a project from the Kili graphical user interface or programmatically.

Follow this recipe to create a new project from the Kili Python SDK:

πŸ“˜

The recipe shows how to create an image object classification project. For more options, refer to our Python SDK reference.

For information on how to create a project from Kili CLI, refer to CLI


Adding assets to project

In Kili, an asset can be a file or a document. This could be a photograph, a satellite image, a video, a PDF, an email, etc. Refer to Supported file formats.

You can add assets located on:

πŸ“˜

For information on how to add assets located on your local servers, refer to Adding assets located on premise.

You can also add assets programmatically.

For information on adding assets through Kili SDK , refer to our example recipe:

Or follow examples in our Python SDK Github repo.

For information on how to add assets through the Kili CLI, refer to CLI reference.

πŸ“˜

  • The maximum number of assets in a project is 25,000.
    If you need more, contact us at [email protected].
  • For video projects, to have a smooth interface, work on videos with less than 1,000 frames.

For more details, refer to Asset lifecycle in Kili app.


Building project ontology

In Kili, labeling (or annotation) jobs are labeling tasks which are associated with specific tools.
For example, each one of these can be considered a Kili labeling job:

  • Classification task with a multi choice dropdown
  • Object detection task with polygon tool
  • Named entities recognition task

Kili jobs may contain nested subjobs, for example:

  • Classification tasks with nested conditional questions
  • Object detection tasks with nested transcription containing additional details

Some jobs types may be interdependent. For example, available relations in a named entities relation job will depend on how classes were defined in a named entities recognition job.

Additionally, each job can be either required or optional.

πŸ“˜

Creating projects with lots of labeling jobs is not efficient. Project complexity and potential performance issues mean that it is harder for members to complete all the jobs. Such a project is also harder to review.
It is better and more efficient to split your jobs in multiple projects, and then to concatenate your labels later, even if this operation must be done outside Kili.


Kili provides the following machine learning job types:

Availability of specific job types depends on your project asset type:


Available labeling job types per asset type

Labeling job typeImageTextPDFVideoVideo (legacy)
Classificationβœ“βœ“βœ“βœ“βœ“
Object detectionβœ“βœ“βœ“
Object relationβœ“βœ“βœ“
Named entities recognitionβœ“βœ“
Named entities relationβœ“βœ“
Transcriptionβœ“βœ“βœ“βœ“βœ“

You can customize your interface through the UI-based interface builder or through JSON settings.


Configuring quality management metrics

Kili's built-in automatic workload distribution mechanism facilitates load management, allows for faster project annotation, and prevents duplicating work.
For example, if several users collaborate on an annotation project, Kili app distributes the data to be annotated to each project member so that each annotator processes unique data. The same data will not be annotated twice, unless Consensus is activated.

To help labelers and reviewers meet the labeling requirements for the project, you can set labeling instructions. Your instructions can also serve as reference, if questions arise.

Labelers who are unsure about a specific label can ask a question. Reviewers, team managers and project admin will be notified and will have the option to answer and close the question.
The number of open questions is shown on the issue button.

For information on how to handle questions asked by labelers, refer to Handling questions and issues.

To further increase your label quality, use the following tools:


Adding users to project and assigning their roles in the project

Users, often called Members in the Kili app, are managed at two levels:

  • Organization they belong to
  • Projects they work for

On Kili Technology platform, a user (identified by email) can only belong to a single organization.
Inside an organization, users can be part of multiple projects, with equal or different roles.

πŸ“˜

Organization admins are admins of all the projects within an organization.

For additional information, refer to:

To manage project members, go to project Settings page and select Members.

From here you can add members of an existing organization to the project or remove them from the project.

The easiest way to add users to a project is to type their email address in the Search user field.

To add a new user to your project:

  1. Click Add member.
  2. Type user's email address.
  3. Specify user's role.
  4. Click Submit.

πŸ“˜

To add a group of users, enter all their email addresses.

Depending on user role, users will have access to different features. Refer to the following:

For information on how to add project members using our SDK, refer to Python SDK.

You can also, quickly copy the code from our recipe here:

Invitees will be informed by email that they were added to a project. If they are invited for the first time, they first have to sign up to join the organization.

πŸ“˜

Maximum number of members per project is limited to 50. If you need more, contact us at [email protected].


Reviewing labeled assets

You can review assets in two ways:

In both cases, the review interface is similar to the labeling interface and allows you to review labeled assets and then:

To be able to quickly find any asset, you can use Kili's advanced filtering features.

To provide feedback to labelers, reviewers can add issues.
The number of open issues is shown on the issue button and is also available from the Analytics page.

For information on how to find, add, and resolve issues, refer to Handling questions and issues.


Analyzing project KPIs

You can generate specific KPIs at fine and complementary levels of granularity, to optimize quality control.

You can access quality KPIs at 2 levels:


Exporting annotations

  1. From the project Queue page, select the assets that you want to export. Quick actions menu appears above the asset list.
  2. From the Quick actions menu, select Export.
  3. From the "Export data" popup window, select your export parameters and then click Export.
  4. Wait for the notification to appear in the top-right corner of the screen.



From the "Export data" popup window, you can customize these export parameters:

12581258

For more information refer to Kili data format.


Did this page help you?