Direct-to-storage uploads

The problem: a 2 GB file through the API

Snip launches Snip Drop: upload a file, get a short link (snip.link/d/x7Kp) that downloads it. Files can be up to 2 GB. The first version accepts uploads through the API: the browser sends the file to an app server, which streams it to object storage.

At 2 GB, that's minutes per upload, each holding an app-server connection and its bandwidth. A burst of uploads starves the redirects running on the same servers (03.12 Tail latency and why p99 matters), a deploy kills every upload in flight, and the load balancer's request size and timeout limits are suddenly a product feature. The app servers add nothing: they just copy bytes from one network socket to another.

Presigned uploads

Let the browser send the bytes directly to object storage, and keep the app in charge of who may upload what:

The presigned URL carries a signature made with Snip's storage credentials over the method, key, expiry and some headers. In Amazon's description, presigned URLs grant time-limited access to objects without changing the bucket's policy, and they are bearer tokens: anyone holding the URL can use it, within the permissions of whoever created it.

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { randomUUID } from "node:crypto";

const s3 = new S3Client({ region: "eu-west-2" });

/** Called by the API after checking the user may upload. The browser PUTs the file straight to S3. */
export async function createUpload(ownerId: number, contentType: string, bytes: number) {
  if (bytes > 2 * 1024 ** 3) throw new Error("too large for Snip Drop");
  const key = `drops/${ownerId}/${randomUUID()}`; // our name, never the user's filename
  const url = await getSignedUrl(
    s3,
    new PutObjectCommand({ Bucket: "snip-drops", Key: key, ContentType: contentType, ContentLength: bytes }),
    { expiresIn: 300 }, // five minutes to start the upload
  );
  return { key, url };
}

Generated with test credentials (no network needed), the URL pointed at the bucket with the chosen key, carried X-Amz-Expires=300, and signed both host and content-length, so an upload of a different size is rejected; an oversized request was refused before any URL was made.

Big files: multipart and resumable

One 2 GB PUT that fails at 95% starts again from zero. Multipart upload splits the file into parts uploaded separately (each with its own presigned URL) and combined at the end:

• Parts upload in parallel, for throughput, and a failed part is retried on its own.
• An upload can be paused and resumed later; it has no expiry until completed or aborted.
• S3 recommends multipart for objects of 100 MB or more. Parts are 5 MiB to 5 GiB, up to 10,000 parts, and the last part may be smaller.
• Abandoned uploads keep their parts, and are billed, until aborted. Set a lifecycle rule to abort incomplete multipart uploads after a few days.

For phones on flaky networks, resumable protocols such as tus make "carry on from where I stopped" part of the protocol: the client asks the server how many bytes it has (a HEAD request returning Upload-Offset), then sends the rest with PATCH.

Don't trust the upload

The bytes arrive without passing your code, so check them afterwards:

• Choose the key yourself. Never use the user's filename as the storage path.
• Quarantine first. Uploads land in a prefix that isn't served; a worker checks size, real type (from the file's bytes, not the declared content type), and malware, then marks it ready.
• Short expiry, exact size. Five minutes to start, the size signed into the URL, and a per-user quota checked when issuing it.

Recap

Interview questions