Data retention, deletion and privacy

The problem: "delete my account"

A Snip customer in Germany closes their account and asks for their data to be erased. Under the EU's GDPR, Snip must erase personal data without undue delay when one of the listed grounds applies, such as the data no longer being needed or consent being withdrawn. "Delete the row in users" is the easy 1%. The customer's data is also in:

• the links and links_by_owner shards (05.01 Sharding: range, hash, directory) and every Redis cache entry for their links;
• the search index (10.03 Keeping a search index in sync) and the analytics rollups and click archives (11.06 Raw events vs derived data);
• their uploaded files in object storage, and copies at CDN edges (13.05 Signed URLs and access control);
• application logs, support tickets, the data warehouse;
• and every backup taken in the last 90 days (14.07 Backups, RPO and RTO).

Collect less, keep it shorter

The cheapest data to delete is data never stored:

• Minimise: Snip stores a click's country and a salted hash of the IP address, not the IP itself; it doesn't need the address to count unique visitors (11.03 HyperLogLog).
• Retention schedules per data type, enforced automatically: raw click events 13 months, application logs 30 days, support tickets 2 years after closure. Expiry by TTL or partition drops (dropping a whole month's partition) is far cheaper than deleting row by row.
• Separate identity from activity: analytics keyed by an internal ID that maps to a person only through one table, which is easy to delete.

A deletion pipeline

Treat erasure like any other change that must reach many systems: as an event that every system consumes (07.06 The outbox pattern and change data capture).

1. Soft delete immediately: mark the account deleted, block sign-in, stop serving its links (and purge them from caches and the CDN).
2. Grace period: keep the data briefly, say 14 days, for accidental closures and fraud checks.
3. Publish account.erased: every service that holds personal data consumes it and deletes or anonymises its copy, then reports back.
4. Track to completion: a deletion ledger records which systems have confirmed, with alerts for stragglers.
5. Let backups age out: backups aren't edited one record at a time; they expire on their own schedule, and if one is ever restored, the deletion ledger is replayed against it before use.

Google Cloud documents the same shape: a deletion request, soft deletion, logical deletion from active systems (generally about two months), then expiry from backups, with a commitment to delete customer data within about six months.

Crypto-shredding: deleting what you can't reach

Some copies are impractical to edit: backups, archives, logs replicated to many places. Crypto-shredding solves this by encrypting each user's sensitive data with a key of its own, and deleting the key to erase the data everywhere at once:

import { createCipheriv, createDecipheriv, randomBytes } from "node:crypto";

const keys = new Map<number, Buffer>(); // in production: a key store, one key per user

export function encryptFor(userId: number, plaintext: string) {
  let key = keys.get(userId);
  if (!key) keys.set(userId, (key = randomBytes(32)));
  const iv = randomBytes(12);
  const c = createCipheriv("aes-256-gcm", key, iv);
  const body = Buffer.concat([c.update(plaintext, "utf8"), c.final()]);
  return Buffer.concat([iv, c.getAuthTag(), body]).toString("base64");
}

export function decryptFor(userId: number, blob: string) {
  const key = keys.get(userId);
  if (!key) throw new Error("key destroyed: data is unreadable");
  const b = Buffer.from(blob, "base64");
  const d = createDecipheriv("aes-256-gcm", key, b.subarray(0, 12));
  d.setAuthTag(b.subarray(12, 28));
  return Buffer.concat([d.update(b.subarray(28)), d.final()]).toString("utf8");
}

export const forgetUser = (userId: number) => keys.delete(userId); // every copy, everywhere, becomes unreadable

Tested: a record decrypted correctly; after forgetUser, the same stored ciphertext could no longer be read. The key store must itself be backed up carefully, without keeping deleted keys, and it becomes critical infrastructure: lose it, and you lose every user's data.

Recap

Interview questions