Module 2 · Lesson 7 of 8
Latency numbers to know
How long memory, disk and network actually take.
Should Gatepass cache event pages in memory? Is one extra database call per request a problem? Is it worth running a server in Europe for students abroad? Each of these is a question about time, and you can answer most of them in your head if you know roughly how long a handful of basic operations take.
Why bother memorising
You've already seen that a request across an ocean spends most of its time in handshakes, and that an index turns a 2-second scan into a millisecond lookup. Both of those insights come from the same skill: knowing the rough cost of each step, so you can add them up and spot the expensive one.
You don't need precise numbers. Hardware changes every year, and the exact figure depends on the machine. What stays stable is the ratios: memory is about a thousand times faster than an SSD, which is several times faster than a network round trip in the same building, which is hundreds of times faster than a round trip across the world.
Learn the ladder, not the decimals. Most design decisions come down to which rung each step of a request is on.
The ladder
Here are the rungs worth knowing, rounded to easy numbers:
- L1 cache read: ~1 ns. Data already on the CPU core.
- Main memory (RAM) read: ~100 ns. Anything in your process's memory, or in Redis on the same machine.
- SSD random read: ~100 µs. Reading a page from disk that isn't cached in memory. A thousand times slower than RAM.
- Round trip within a data centre: ~0.5 ms. Your server asking the database or cache on another machine.
- Spinning disk seek: ~5 ms. Rare for databases now, but still around for cheap bulk storage.
- Round trip between continents: 100–250 ms. Mumbai to Frankfurt is about 130 ms, Mumbai to California about 230 ms.
Units first, since they trip everyone up: a millisecond (ms) is a thousand microseconds (µs), and a microsecond is a thousand nanoseconds (ns).
See it at human scale
Nanoseconds are impossible to feel. So stretch time: pretend one nanosecond lasts one full second, and keep every ratio the same. Flip the toggle and watch what happens to the bottom of the ladder.
Interactive
How long things take
Round trip in a data centre takes as long as 5K RAM reads.
In stretched time, a memory read takes under two minutes. A database round trip in the same data centre takes almost six days. A request from Mumbai to a server in California takes over seven years. Every rung down is a different world.
What this means for Gatepass
Lay one Gatepass "event page" request along the ladder:
Reading this ladder answers the questions from the start:
- Should event pages be cached in memory? A cache hit saves the 1.5 ms database trip. For a single student, that's invisible next to 12 ms of network. The real win is load: on sale day, the database stops answering the same query thousands of times a second. That's Module 3.
- Is one extra database call a problem? One, no: half a millisecond. Fifty in a loop, yes: 25 ms of pure waiting, more than the network. This is the N+1 query bug, and the ladder tells you exactly why it hurts.
- Should there be a server in Europe? For students in Germany, every round trip to Mumbai is ~130 ms, and a fresh connection needs three of them. A nearby server (or at least a nearby edge that terminates TLS) cuts hundreds of milliseconds. Nothing you do to the code comes close.
Rules of thumb
A few rules fall straight out of the ladder:
- Memory beats disk beats network. If something is read often, keep it as far up the ladder as you can.
- Avoid round trips before optimising code. Shaving 10% off code that takes 1 ms saves 0.1 ms. Removing one cross-continent round trip saves 130.
- Distance is physics. Light in fibre needs about 5 ms per 1,000 km each way. No software fixes that; only moving the work closer does.
- Sequential reads are cheap, random ones aren't. Reading 1 MB in a row from an SSD is fast; reading 256 scattered 4 KB pages is slow. Databases, logs and Kafka are all designed around this.
Check yourself
The booking endpoint makes 20 sequential calls to Postgres in the same data centre. Roughly how much time do the round trips alone add?
Key takeaways
- Learn the ratios: RAM ~100 ns, SSD ~100 µs, same data centre ~0.5 ms, across continents 100–250 ms.
- Each rung down is roughly 10–1,000× slower than the one above.
- In most requests, network round trips dominate; count them before tuning code.
- Sequential round trips multiply; batch them or run them in parallel.
- Distance costs about 5 ms per 1,000 km each way, and only moving closer reduces it.
Finished reading?
Mark it complete to tick it off in the course.