Why memorise any numbers?
You can't judge a design without a sense of scale. Is one more network hop a big deal? Should this lookup hit memory or disk? Will five years of data fit on one machine? A few numbers, known to the nearest power of ten, answer most of these questions in your head.
You don't need exact values. Hardware changes every year. What lasts are the ratios: memory is about a thousand times faster than an SSD read, and a round trip across the world is hundreds of times slower than one inside a datacenter.
Latency
These are the classic "latency numbers every programmer should know", first published by Jeff Dean and Peter Norvig around 2012 and updated since. Tick the box to see them on a human scale.
| Read from the CPU's L1 cache | 1 ns |
| Read from main memory (RAM) | 100 ns |
| Read 1 MB sequentially from memory | 3 µs |
| Random read from an SSD | 16 µs |
| Read 1 MB sequentially from an SSD | 49 µs |
| Round trip inside one datacenter | 500 µs |
| Spinning disk seek | ~3 ms |
| Round trip California → Netherlands → California | 150 ms |
Bars use a log scale: each step is ten times longer. Values from Colin Scott's interactive latency page, which projects the original list forward to 2020.
What to take away:
- Memory is fast; everything else is far away. Reading RAM takes about 100 ns. That's why caches (memory) sit in front of databases (disk), as you'll see in 06.01 Where caches live.
- A network hop inside a datacenter costs about half a millisecond. A request that calls ten services one after another has spent 5 ms just travelling.
- Crossing an ocean costs about 150 ms, and the speed of light means no hardware upgrade fixes it. The only fix is to put data closer to users, which is what CDNs and multi-region designs do.
- Sequential beats random. Reading data that sits together is far cheaper than jumping around. Databases and logs are built around this.
Sizes
Data sizes go up in steps of about a thousand. Powers of two line up with them closely enough for estimates:
| Power of two | Roughly | Name |
|---|---|---|
| 2¹⁰ | a thousand (10³) | kilobyte (KB) |
| 2²⁰ | a million (10⁶) | megabyte (MB) |
| 2³⁰ | a billion (10⁹) | gigabyte (GB) |
| 2⁴⁰ | a trillion (10¹²) | terabyte (TB) |
| 2⁵⁰ | a quadrillion (10¹⁵) | petabyte (PB) |
And the sizes of everyday things:
| Thing | Size |
|---|---|
| A character of English text | 1 byte |
| A 64-bit number or timestamp | 8 bytes |
| A UUID | 16 bytes (36 as text) |
| A tweet-sized post with its metadata | about 1 KB |
| A compressed photo | a few hundred KB to a few MB |
| A minute of HD video | tens of MB |
Time
Estimates usually turn "per day" into "per second". Two conversions do most of the work:
- A day has 86,400 seconds. Call it 100,000 (10⁵). Then a million events a day is about 10 a second.
- A month has about 2.5 million seconds, and a year about 30 million.
Rounding a day up to 100,000 seconds makes your per-second numbers about 15% low. That's fine for an estimate, and you can say so.
Which is slower: reading 1 MB sequentially from an SSD, or one round trip to another server in the same datacenter?
Recap
Interview questions
Why put a cache in front of a database? Answer with rough numbers.