DocumentationRecipesReferenceGraphQLChangelog
Log In

Generating non-expiring signed URLs on AWS

Amazon S3 has a maximum signature expiry of 7 days. To upload signed URLs that don’t expire, we recommend proxying URLs through an endpoint on your server.

This example demonstrates how to proxy through a simple flask app. We provided a one-click deploy through Heroku, but you can also build this simple handler into your existing web service.

Step 1: Deploy a proxy endpoint

This endpoint accepts a signed URL with our JWT secret and returns a new, signed s3 URL to an asset.

First, you’ll need to get IAM information to be able to create pre-signed URLs:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • a bucket name

📘

Make sure that this IAM user can LIST and GET files in the bucket.

Have a look at the example proxy that we prepared

You can deploy it with one click here

Step 2: Generate signed URLs pointing at our proxy

For each asset in our s3 bucket, we will generate a signed URL with our JWT secret that points to our server endpoint.

  1. From Heroku, click open app to get the host URL of your new app. Then, get the generated secret (settings > reveal config vars).

  2. Clone the generate-tokenized-urls repository:

git clone https://github.com/kili-technology/generate-tokenized-urls
cd generate-tokenized-urls/
  1. Create your URLs:
// confirm you have node.js installed
node --version

npm install
node cli.js
  --bucket <your-aws-bucket-name>
  --host https://<your-new-heroku-url>.herokuapp.com/
  --secret <heroku-generated-config-secret>
  --output kili-import.json

This code creates the kili-import.json file. Import the URLs from kili-import.json to Kili and you’re good to go.


Did this page help you?