Adding assets to project

You can add assets as part of the project creation process or later on in your project lifetime.

You can add assets from these sources:

• Your local workstation
• Remote storage (list of links)
• Remote storage (integration)

📘

For information on how to add assets located on your local servers, refer to Adding assets located on premise.

You can also add assets programmatically.

For information on adding assets through Kili SDK , refer to our example recipe:

Or follow examples in our Python SDK Github repo.

For information on how to add assets through the Kili CLI, refer to CLI reference.

📘

• The maximum number of assets in a project is 25,000.
• For video projects, to have a smooth interface, work on videos with less than 1,000 frames

Adding rich text assets

Importing rich text is currently only supported through the Kili API. Refer to an example tutorial.

Adding video assets

For video data, Kili Technology lets you decide if you want to import your data as native video and annotate them as video or extract the video into a list of images (frames) and annotate them as individual images. For more information, refer to Labeling video assets.

Adding assets from your local workstation

1. If you're adding assets to an existing project, from your project queue, click the More button, and select Upload data.
2. Select Hosted by Kili.
3. Drag & drop your asset files. Use one asset per file, max 500 files per batch.

Your data will be securely stored in a dedicated bucket on our cloud. Only you have access to it.

Adding assets located in remote storage (list of links)

Go to your project and then:

1. If you're adding assets to an existing project, from your project queue, click the More button, and select Upload data.
2. Select From remote storage.
3. Drag & drop the CSV file with URLs of the assets. Refer to Required CSV file format.

Your data remains hosted at its origin. No data is stored on Kili servers.

Required CSV file format

Create a CSV file with two columns: externald and url. Use one asset per line. Refer to this example:

externalId,url
image_1,https://images.pexels.com/photos/45209/purple-grapes-vineyard-napa-valley-napa-vineyard-45209.jpeg

Adding assets located in remote storage (integration)

📘

• You must first create an integration. For details, refer to Creating a remote storage integration.
• By using integrations, you can add image and text assets to Kili. TIFF and GeoTIFF images are not supported yet.

To add assets located in remote storage through an integration:

1. If you're adding assets to an existing project, from your project queue, click the More button, and select Upload data.
2. Click New.
3. From the Select remote storage list, select your existing integration.
4. Click Save and synchronize.
5. If changes between your storage and the project are detected, a button will be displayed so you can complete the upload of assets. Otherwise "no updates" is displayed on the data connector.

Creating a remote storage integration

❗

Beta feature

Remote storage integrations are currently a beta feature. If you encounter any issues, please report them to us at [email protected]

📘

• Only organization admins can manage remote integrations.
• Kili supports integrations with AWS S3, Azure Blob Storage, and Google Cloud Platform.
• An integration can be shared by multiple projects.

Follow the steps listed in either of:

• Creating an Azure Blob Storage integration
• Creating an Amazon Web Storage integration
• Creating a GCP Cloud Storage intergation

Adding assets located on premise

📘

When you import video assets, Kili runs some backend checks on the files (for example, to determine the frame rate). In on-prem configurations where the Kili app and video files are hosted in different places and Kili backend has no access to these files the video upload fails.

The data will be streamed through the web browser and only available within your internal network.
If you are planning to use data located on your hard disk, for security reasons, you must use HTTPS with certificates certified by an authority. This means that you need a DNS which points to your files served locally. Follow this process:

1. Set up your DNS.
2. Generate a certificate.
3. Create your server.
4. Create a CSV file containing the URLs to the data you want to serve. Refer to Required CSV file format.
5. Use the CSV file to add assets to your project. Refer to Adding assets from the Web.

🚧

• If the local server is down, you will lose access to your data while using Kili Technology.
• To provide external users access to the data and to set up localhost as your file server, it's necessary to display your localhost on the Internet. Follow this tutorial: Exposing localhost server to the Internet.
• For security reasons, we avoid mixed content and enforce assets to be served over HTTPS (not HTTP). So all asset URLs must begin with <https://>.

Setting up your DNS

Setting up your DNS depends on the service that you use. For example, if you use Wordpress, from the <https://wordpress.com/domains/manage> page, create a domain name that points to your file server. The example domain that we will be using is: test-ssl.kili-technology.com.

Generating a certificate

Follow this procedure to generate a certificate that is signed by an authority:

1. Run the following command:

certbot -d test-ssl.kili-technology.com --manual --logs-dir certbot --config-dir certbot --work-dir certbot --preferred-challenges dns certonly

📘

If certbot is missing, you can add it using your system's package manager.

2. When prompted, add an email address.
   A message appears, listing the value that you must add to the subdomain:

Please deploy a DNS TXT record under the name:
_acme-challenge.test-ssl.kili-technology.com
with the following value:
<example-value>

3. Log into your DNS provider and add the <example-value> to the provided subdomain. In our example, we're using _acme-challenge.test-ssl.kili-technology.com.

4. Press Enter. Two files are created: fullchain.pem and privkey.pem.

Creating your server

Create a local server and add your files to it.

Here is an example of how to create a local server using python:

import ssl, os
from http.server import HTTPServer, SimpleHTTPRequestHandler
class Handler(SimpleHTTPRequestHandler):
    def send_response(self, *args, **kwargs):
        SimpleHTTPRequestHandler.send_response(self, *args, **kwargs)
        self.send_header('Access-Control-Allow-Origin', '*')
port = 443
httpd = HTTPServer(('0.0.0.0', port), Handler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile='privkey.pem', certfile='fullchain.pem', server_side=True)
print("Server running")
httpd.serve_forever()

Save this code as server.py and place it in the same folder as fullchain.pem and privkey.pem. Then, run your server using the sudo python server.py command.

Learn more

For an end-to-end example of how to programmatically add assets, asset metadata, and asset pre-annotations to a project using Kili's Python SDK, refer to our Importing assets and labels tutorial. We've also created a separate tutorial on adding video assets to Kili.