sehaj/learn
Docker/07 Production

07.01 · with code · 12 min

Run as non-root

Containers run as root unless you say otherwise. Drop to a normal user, then strip capabilities, privilege escalation and write access until there's almost nothing left to abuse.

The core idea

Unless an image sets USER, its process runs as root (UID 0). Namespaces limit what that root can see, but it's still UID 0 talking to the same kernel as the host. If there's ever a container escape bug, root inside is much closer to root outside. Running as a normal user is the cheapest, biggest security win available, and Docker gives you four more layers to stack on top.

Who are you in there?

❯terminal
$ docker run --rm alpine id
uid=0(root) gid=0(root) groups=0(root),0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),...
$ docker run --rm --entrypoint id notes-api
uid=1000(node) gid=1000(node) groups=1000(node)

The notes API has run as node since its multi-stage build (02.04 Multi-stage builds), thanks to USER node in the final stage. The official Node images create that user for exactly this. Root can rewrite system files in its container; node can't:

❯terminal
$ docker run --rm alpine sh -c 'echo x > /etc/passwd-test && echo root-can-write-etc'
root-can-write-etc
$ docker run --rm --entrypoint sh notes-api -c 'echo x > /etc/passwd-test'
sh: 1: cannot create /etc/passwd-test: Permission denied

If your base image has no suitable user, create one:

Dockerfile
RUN groupadd --system app && useradd --system --gid app --no-create-home app
USER app

Prefer a numeric ID (USER 1000) when you can: Kubernetes can only verify "not root" from a number, not a name.

Layer 2: drop capabilities

Linux splits root's powers into about 40 capabilities: changing file ownership, binding low ports, tracing processes and so on. Docker gives containers a reduced default set; --cap-drop ALL removes the rest:

❯terminal
$ docker run --rm alpine grep CapEff /proc/self/status
CapEff: 00000000a80425fb
$ docker run --rm --cap-drop ALL alpine grep CapEff /proc/self/status
CapEff: 0000000000000000
$ docker run --rm alpine chown nobody /etc/hostname && echo chown-ok
chown-ok
$ docker run --rm --cap-drop ALL alpine chown nobody /etc/hostname
chown: /etc/hostname: Operation not permitted

That hex value is a bitmap of capabilities; all zeros means none. Even root inside can't chown without CAP_CHOWN. Most apps need none at all; add back only what fails, with --cap-add.

Layer 3: no new privileges

--security-opt no-new-privileges stops a process from gaining privileges after it starts, for example through a setuid binary like su or sudo. It closes the classic "get a shell as a normal user, then escalate" path.

Layer 4: read-only filesystem

From 04.04 tmpfs mounts: --read-only plus tmpfs for the paths an app must write. The notes API writes nothing to disk, so it needs no tmpfs at all.

All together

❯terminal
$ docker run --rm --read-only --cap-drop ALL --security-opt no-new-privileges \
-e DATABASE_URL=postgres://x:y@nowhere:5432/z notes-api
database not ready (attempt 1/15), retrying in 2s

It starts normally (and waits for a database that doesn't exist here). Inside the production stack in 07.07 Project: deploy to a VPS, the API's process shows exactly this:

❯terminal
$ docker compose -f compose.prod.yaml exec api grep -E "CapEff|NoNewPrivs" /proc/1/status
CapEff: 0000000000000000
NoNewPrivs: 1

Images that start as root on purpose

Postgres and Redis images start as root, prepare their data directories, then switch to their own user before running the server. docker exec would show you root (that's the default user for exec), but the main process isn't:

❯terminal
$ docker top notes-prod-db-1 -o pid,user,args
PID USER ARGS
... 999 postgres

For Nginx, use nginxinc/nginx-unprivileged, which runs as the nginx user throughout. It's what the production stack uses.

Predict first

An attacker gets code execution inside the notes API container (USER node, --cap-drop ALL, --read-only, no-new-privileges). Which of these can they still do?

Commit to an answer before you read on.
CheckpointHarden and prove it

Run the notes API with all four layers and check /proc/1/status for CapEff and NoNewPrivs. Try touch /app/x through docker exec. Then run plain nginx:alpine with --cap-drop ALL and read why it fails; compare with nginxinc/nginx-unprivileged:alpine.

STUCK? ▸

An app that fails with --cap-drop ALL usually logs Operation not permitted next to the action it tried. Add back that one capability, e.g. --cap-add NET_BIND_SERVICE to bind a port below 1024 on older Docker versions.

Recap

Interview questions

MidfreqMust-knowConcept

Why shouldn't containers run as root, and how do you harden one?

Report a mistakeEdit this page