Generating non-expiring signed URLs on AWS

Amazon S3 has a maximum signature expiry of 7 days. To upload signed URLs that don’t expire, we recommend proxying URLs through an endpoint on your server.

This example demonstrates how to proxy through a simple flask app. We provided a one-click deploy through Heroku, but you can also build this simple handler into your existing web service.

Step 1: Deploy a proxy endpoint

This endpoint accepts a signed URL with our JWT secret and returns a new, signed s3 URL to an asset.

First, you’ll need to get IAM information to be able to create pre-signed URLs:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
a bucket name

📘
Make sure that this IAM user can LIST and GET files in the bucket.

When your proxy is ready, you can deploy it with one click here:

Step 2: Generate signed URLs pointing at our proxy

For each asset in our s3 bucket, we will generate a signed URL with our JWT secret that points to our server endpoint.

From Heroku, click open app to get the host URL of your new app. Then, get the generated secret (settings > reveal config vars).
Clone the generate-tokenized-urls repository:

git clone https://github.com/kili-technology/generate-tokenized-urls
cd generate-tokenized-urls/

Create your URLs:

// confirm you have node.js installed
node --version

npm install
node cli.js
  --bucket <your-aws-bucket-name>
  --host https://<your-new-heroku-url>.herokuapp.com/
  --secret <heroku-generated-config-secret>
  --output kili-import.json

This code creates the kili-import.json file. Import the URLs from kili-import.json to Kili and you’re good to go.