sehaj/learn
Docker/04 Data

04.01 · with code · 8 min

The writable layer is temporary

Why anything a container writes disappears with it, what survives a restart and what doesn't, and the three places data can live instead.

The core idea

Everything a container writes, unless you say otherwise, goes into its writable layer. That layer belongs to the container: it survives stop and start, and it's deleted with docker rm. Containers are meant to be thrown away and replaced, so anything you want to keep has to live outside the container.

Watch data disappear

❯terminal
$ docker run -d --name scratch alpine sleep 1000
$ docker exec scratch sh -c 'echo "important" > /data.txt; dd if=/dev/zero of=/big bs=1M count=50'
$ docker ps -s --filter name=scratch --format 'table {{.Names}}\t{{.Size}}'
NAMES SIZE
scratch 52.4MB (virtual 61.8MB)
$ docker diff scratch
A /big
A /data.txt
$ docker stop scratch && docker start scratch
$ docker exec scratch cat /data.txt
important
$ docker rm -f scratch
$ docker run --rm alpine cat /data.txt
cat: can't open '/data.txt': No such file or directory

docker ps -s shows the writable layer's size (52.4 MB) and, in brackets, that plus the image underneath. A stop and start keeps the file. Removing the container deletes it, and a new container from the same image starts clean.

That's fine for a cache. It's a disaster for a database, which is exactly what happens if you run Postgres and later recreate the container to upgrade it.

Why not just keep the container forever?

Because you can't, in practice:

  • Deploying means replacing. A new image version means a new container.
  • Writable layers are slow for heavy writes. Changing a file from the image first copies it into the writable layer (copy-on-write), and the storage driver adds overhead that databases feel.
  • Writable layers are hard to back up, move to another server or share between containers.

Three places data can live

Where a container's files can come from
Inside the containerdies with it
writable layer
image layers (read-only)
Mounted from outsideoutlives it
volume
bind mount
tmpfs
A mount replaces one path inside the container with storage that Docker manages separately from the container.
MountLives inUse it forLesson
Volumestorage Docker manages (/var/lib/docker/volumes/…)databases, uploads: data the app owns04.02 Volumes
Bind mounta folder you choose on the hostsource code in development, config files04.03 Bind mounts
tmpfsmemory, never diskscratch space, secrets that must never touch disk04.04 tmpfs mounts
Predict first

You run Postgres in a container with no mounts, add data, then docker stop and docker start it. Then you upgrade by docker rm and docker run with the newer image. What happens to the data?

Commit to an answer before you read on.

That quiz is the next lesson's motivation: name your volumes.

CheckpointLose some data on purpose

Run the commands above. Then check docker ps -s on a running notes API: its writable layer should be tiny, because it doesn't write files. That's the goal for most app containers.

STUCK? ▸

If the file is still there after docker rm, check you started a new container with docker run rather than docker start on the old one.

Recap

Interview questions

JuniorfreqMust-knowConcept

Where does data written inside a container go, and what happens to it when the container is removed?

Report a mistakeEdit this page
☰ Data04.01 · 22/44next →