Split the database; scale up or out

The problem: two jobs, one machine

A few months in, Snip has a few thousand users. Once a month an owner downloads a report of every click on every link, and while that query runs, redirects slow from 5 ms to 400 ms. The database and the app are on the same machine, so a heavy query takes CPU, memory and disk bandwidth away from everything else.

The two parts also want different hardware. The app needs CPU and very little disk. The database wants lots of memory (to keep hot data out of the disk) and fast storage. On one machine you can't size either one properly.

Step 1: give the database its own machine

What Snip gains:

• Each part sized for its job. The database gets memory and fast disks; the app gets CPU.
• No more fighting. A busy app can't starve the database, and the other way round.
• The app becomes replaceable. It holds no data, so you can rebuild it, redeploy it, or later run several copies, all pointing at the same database.

What it costs:

• A network hop on every query: about half a millisecond inside one datacenter (01.04 Numbers to know). A redirect that makes one query barely notices; a page that makes 50 queries one after another does.
• One more machine to pay for, patch and monitor.
• Lower availability, for now. A redirect needs both machines. Two machines that are each up 99.9% of the time give about 99.8% together. Splitting things up doesn't remove single points of failure; it just separates them, so you can fix each one on its own. That's the next two lessons.

Step 2: up or out?

When a part runs out of room, there are two ways to grow it:

• Scale up (vertical scaling): move it to a bigger machine, with more cores, memory or faster disks.
• Scale out (horizontal scaling): add more machines and split the work between them.

The usual answer: out for the app, up for the database

Most systems end up doing both, for a simple reason: things that hold no data scale out easily, and things that hold data don't.

• The app keeps nothing between requests (or can be made to, in 02.03 Stateless servers behind a load balancer). Any copy can answer any request, so you add copies and spread requests across them.
• The database holds the data. Splitting it across machines means deciding which machine owns which rows, and every query that spans them gets harder (that's sharding, in 05.01 Sharding: range, hash, directory). So you scale it up for as long as you can, and take load off it in other ways first: a cache and read copies (02.04 Read replicas and a cache).

Modern servers are very large, and a single well-tuned database goes much further than most people expect. Snip's estimate from 01.05 Back-of-the-envelope estimation (1,000 reads and 10 writes a second at peak, 300 GB) fits on a mid-sized database server with years of room.

Don't forget the standby

Scaling up still leaves the database as a single point of failure. The standard fix is a standby: a second database server that receives every change from the main one (the primary) and can take over within seconds to minutes if the primary dies. That swap is called failover.

It doesn't add capacity, and it costs a whole second server that mostly waits. But it means a dead disk is a short blip instead of hours of restoring from backup. You'll see how it works, and what can go wrong, in 04.07 Replication: leaders, followers and lag.

Recap

Interview questions