sehaj/learn
Docker/04 Data

04.05 · build · 25 min

Project: persist Postgres

Move the notes API's data into Postgres on a named volume, prove it survives deleting every container, and practise a backup and restore.

The brief

Notes currently live in the API's memory, so every restart wipes them. Done means:

What changed in the API

Check out the reference (git checkout docker-step-03) or make these changes yourself in project/docker/api:

  • pg, the Postgres client, is now a runtime dependency (npm install pg, and npm install -D @types/pg). For the first time, the final stage's npm ci --omit=dev installs something.
  • DATABASE_URL is required; without it the API exits 1 with a message saying so (03.03 Environment variables and config).
  • At startup it retries SELECT 1 every 2 seconds, up to 15 times, then creates the notes table if it doesn't exist.
  • /healthz still doesn't touch the database (03.07 Healthchecks (project step 2)).
src/server.ts
const pool = new pg.Pool({ connectionString: databaseUrl });

// The database container may still be starting when we start, so retry for a while before giving up.
async function waitForDatabase(attempts = 15) {
for (let i = 1; ; i++) {
  try {
    await pool.query("SELECT 1");
    return;
  } catch (err) {
    if (i === attempts) throw err;
    console.log(`database not ready (attempt ${i}/${attempts}), retrying in 2s`);
    await new Promise((resolve) => setTimeout(resolve, 2000));
  }
}
}

Step 1: a network and a volume

The API reaches Postgres by container name, notes-db. That works on a user-defined network, which gives containers DNS names for each other. You'll learn how in 05.02 User-defined bridges and DNS; for now, create one:

❯terminal
$ docker network create notes-net
$ docker volume create notes-pgdata
notes-pgdata

Step 2: start both, API first

Start the API before the database on purpose, to watch it wait:

❯terminal
$ docker build -t notes-api .
$ docker run -d --name notes --network notes-net -p 8080:3000 \
-e DATABASE_URL=postgres://notes:notes@notes-db:5432/notes notes-api
$ docker run -d --name notes-db --network notes-net \
-e POSTGRES_USER=notes -e POSTGRES_PASSWORD=notes -e POSTGRES_DB=notes \
-v notes-pgdata:/var/lib/postgresql postgres:18
$ docker logs notes
database not ready (attempt 1/15), retrying in 2s
database not ready (attempt 2/15), retrying in 2s
notes-api listening on :3000 (max note length 280)
$ curl -X POST localhost:8080/notes -H 'content-type: application/json' -d '{"text":"this one should survive"}'
{"id":"ca767c14-0820-4cd5-963d-9df6240ada88","text":"this one should survive","createdAt":"2026-11-02T20:32:26.604Z"}

The POSTGRES_* variables only matter the first time, when the volume is empty and the image initialises a database.

The Postgres 18 mount path

Look closely at -v notes-pgdata:/var/lib/postgresql. Most tutorials, and every Postgres image before 18, mount /var/lib/postgresql/data. Postgres 18 images moved the data into a version-specific folder (/var/lib/postgresql/18/docker) and declare the parent as the volume. Use the old path with 18, and it refuses to start:

❯terminal
$ docker run -d --name oldpath -e POSTGRES_PASSWORD=x -v oldpath-data:/var/lib/postgresql/data postgres:18
$ docker inspect -f 'status={{.State.Status}} exit={{.State.ExitCode}}' oldpath
status=exited exit=1
$ docker logs oldpath
Error: in 18+, these Docker images are configured to store database data in a
format which is compatible with "pg_ctlcluster" (specifically, using
major-version-specific directory names).
...

Output trimmed. That error is a gift: on older setups, a wrong path silently put your data in an anonymous volume instead (04.02 Volumes). Always check the image's docs for where it keeps data, or run docker image inspect -f '{{json .Config.Volumes}}' IMAGE.

Step 3: delete everything, keep the data

❯terminal
$ docker rm -f notes notes-db
$ docker volume ls --filter name=notes-pgdata
DRIVER VOLUME NAME
local notes-pgdata
$ docker run -d --name notes-db --network notes-net \
-e POSTGRES_USER=notes -e POSTGRES_PASSWORD=notes -e POSTGRES_DB=notes \
-v notes-pgdata:/var/lib/postgresql postgres:18
$ docker run -d --name notes --network notes-net -p 8080:3000 \
-e DATABASE_URL=postgres://notes:notes@notes-db:5432/notes notes-api
$ curl localhost:8080/notes
[{"id":"ca767c14-0820-4cd5-963d-9df6240ada88","text":"this one should survive","createdAt":"2026-11-02T20:32:26.604Z"}]
$ docker logs notes-db 2>&1 | grep -i initialization
PostgreSQL Database directory appears to contain a database; Skipping initialization

Both containers are brand new. The note survived because it never lived in a container.

Predict first

You recreate notes-db with -e POSTGRES_PASSWORD=different but the same notes-pgdata volume. The API still uses the old password. What happens?

Commit to an answer before you read on.

Step 4: back up and restore

Copying the volume's files while Postgres is running can capture a half-written state. Use pg_dump, which runs inside the database container and writes a consistent SQL snapshot to your machine:

❯terminal
$ docker exec notes-db pg_dump -U notes notes > backup.sql
$ grep -A2 "^COPY public.notes" backup.sql
COPY public.notes (id, text, created_at) FROM stdin;
ca767c14-0820-4cd5-963d-9df6240ada88 this one should survive 2026-11-02 20:32:26.604687+00
\.

A backup you haven't restored is a hope, not a backup. Restore it into a completely separate database:

❯terminal
$ docker run -d --name restore-db -e POSTGRES_USER=notes -e POSTGRES_PASSWORD=notes -e POSTGRES_DB=notes \
-v notes-pgdata-restored:/var/lib/postgresql postgres:18
$ docker exec -i restore-db psql -q -U notes notes < backup.sql
$ docker exec restore-db psql -U notes notes -c 'SELECT text FROM notes'
text
-------------------------
this one should survive
(1 row)

Note the -i on the restore: it keeps stdin open so psql can read the file from your machine.

CheckpointProject step 3: persist Postgres
  • Switch the notes API to Postgres, or check out docker-step-03.
  • Run both containers as above, create three notes, then docker rm -f both and recreate them. All three notes should still be there.
  • Dump the database, restore it into a fresh container and volume, and count the rows.
  • Clean up with docker rm -f notes notes-db restore-db and docker volume rm notes-pgdata-restored. Keep notes-pgdata; the next steps use it.
STUCK? ▸

The API exits with could not reach the database: both containers must be on notes-net, and the host in DATABASE_URL must be the container name, notes-db. Empty notes after recreating: check both docker run commands use exactly notes-pgdata:/var/lib/postgresql. Compare with git diff docker-step-03 -- project/docker/api.

What's next

Three long docker run commands with a network, a volume and a password typed twice is a lot to get right by hand. Module 05 explains the network you just used (05.02 User-defined bridges and DNS), and Module 06 replaces all of this with one Compose file (06.06 Project: one-command dev environment).

Interview questions

MidfreqMust-knowDesign

How would you run a database in Docker safely?

Report a mistakeEdit this page

Used in

☰ Data04.05 · 26/44next →