Availability, durability, consistency, latency, cost

Five words that need precise meanings

People say "the system should be reliable and fast" and mean five different things. Separating them matters, because they're bought in different ways and they pull against each other:

Availability

Availability is the share of time (or of requests) the system works. It's quoted in nines: 99.9% is "three nines". Each extra nine cuts the downtime you're allowed by ten:

Figures from the availability table in Google's Site Reliability Engineering book.

Two pieces of arithmetic come up again and again:

• Parts in a chain multiply. If a request needs the app and the database, and each is up 99.9% of the time, the request succeeds about 99.9% × 99.9% ≈ 99.8% of the time. Every hop you add lowers availability.
• Spare copies multiply the failure chance. If either of two independent copies can serve a request, and each is down 1% of the time, both are down only 1% × 1% = 0.01% of the time: 99.99% available. Redundancy is how you buy nines, as long as the copies don't fail together.

Durability

Durability is the promise that data you've accepted won't be lost. It's different from availability: a system can be down for an hour and lose nothing (low availability, high durability), or stay up while quietly losing writes (the reverse, and much worse).

Durability is bought with copies in separate places: several disks, several machines, several buildings. Amazon S3 is designed for 99.999999999% ("eleven nines") durability of objects over a year, but 99.99% availability: the data is almost never lost, though now and then you briefly can't reach it.

Consistency

Consistency here means: after a write, do readers see it? Two broad answers:

• Strong consistency: every read sees the latest write. Simple to reason about, but every copy has to agree before you answer, which costs time and availability.
• Eventual consistency: copies catch up after a short delay, so a read may briefly return old data. Faster and more available, but your design has to tolerate stale reads.

Most systems mix them, feature by feature. Snip needs a new link to work the moment it's created (the person who made it will click it straight away), but click counts can lag by a minute and nobody will notice.

Latency and throughput

Latency is how long one request takes. Throughput is how many requests the system handles per second. They're related but different: a highway can carry thousands of cars an hour (throughput) while each car still takes twenty minutes to drive it (latency).

Never describe latency with an average. Suppose 98 of 100 requests take 20 ms and 2 take 2 seconds. The average is about 60 ms, a number no request actually experienced. Use percentiles instead:

• p50 (the median): half of requests are faster than this. Here, 20 ms.
• p99: 99% are faster than this. Here, 2 seconds. That's the slow tail, and at high traffic it's someone every second.

That's why Snip's requirement reads "under 50 ms at p99", not "fast on average". You'll see why the tail matters so much in 03.12 Tail latency and why p99 matters.

Cost

Cost is money (machines, storage, bandwidth, managed services) and also effort: every extra component is something to deploy, monitor, upgrade and get paged for at 3 a.m. A design that meets every other requirement but needs a team of ten to run it may still be the wrong design.

Recap

Interview questions