Most of the numbers that decide an architecture can be settled on the back of a napkin, before a line of the prototype is written. A handful of formulas and a latency ladder you can recite from memory rule out bad designs a profiler could only catch after they’re built. The same numbers expose the quieter traps — like a redundant trio of replicas that one dead node takes down whole. Let’s do the arithmetic.

The metrics that actually decide a design are few: how long things take (latency), how many we can do per second (throughput), how often they fail (failure rate), and how much headroom we keep. The discipline is to estimate all four before building — lean on arithmetic over tools, and do the math before you reach for a profiler.

Little’s law

The single most useful formula for capacity is Little’s law :

L = λW

The average number of items in a system equals the arrival rate times the time each item spends inside. It is distribution-free — it holds whatever the arrival pattern, service-time spread, or scheduling order, as long as the system is stable (the backlog isn’t growing without bound). That’s what makes it a napkin tool: you never need to know the distributions.

Forward — size a pool. A service answers each request in W = 200 ms and must sustain λ = 500 req/s. Average requests in flight: L = λW = 500 × 0.2 = 100. So the pool — threads, connections, concurrent async tasks — needs at least 100 slots, or it can’t hit the target without queuing.

Backward — find a fixed pool’s ceiling. Fifty slots at 200 ms caps throughput at λ = L / W = 50 / 0.2 = 250 req/s. Push past that and the surplus queues, W climbs, and you spiral. The arithmetic ruled out the under-provisioned design before a single load test ran.

One honest caveat on the napkin: W = 200 ms is a single number standing in for a whole distribution, and real latency distributions are lopsided — reason in percentiles (p50/p99), not means, or the tail will mug you. That’s a companion subject of its own: Statistics for programmers .

A latency ladder you can recite

Little’s law needs a W; the ladder below is where the first guess comes from. Memorize the ratios, not the digits — hardware drifts, but most of the gaps between tiers barely move. The canonical figures (Dean’s and Norvig’s , via Jonas Bonér’s 2012 compilation ):

Operation	Time
L1 cache reference	0.5 ns
Branch mispredict	5 ns
Mutex lock/unlock	25 ns
Main-memory reference	100 ns	≈200× L1
Send 1 KB over 1 Gbps	10 µs
SSD random read (4 KB)	16–150 µs
Read 1 MB from RAM	~0.25 ms
Same-datacenter round trip	0.5 ms
Read 1 MB from SSD	~1 ms	4× RAM
Disk (HDD) seek	10 ms	20× a same-datacenter round trip
Read 1 MB from HDD	20 ms
Round trip cross-continent (CA↔NL)	150 ms

A local file read is a cache hit at RAM speed (page cache) or a miss that drops to the SSD/HDD tier above. Common services, in-datacenter and including the network round trip: a Redis or Memcached GET is sub-millisecond (~0.1–1 ms); an indexed database point query is ~1–10 ms, joins and complex queries 10–100 ms and up; an uncached DNS lookup ~10–100 ms. A cold connection adds a TCP round trip plus a TLS handshake (one round trip for TLS 1.3, two for 1.2).

The ladder has not aged evenly since those 2012 figures. Storage moved the most: an NVMe random read lands near 16 µs where the classic number said 150 µs — an order of magnitude, and the reason the table shows a range. Datacenter links jumped from 1 Gbps to 10–100 Gbps, shrinking the serialization row with them. The ends barely moved: main memory has answered in ~100 ns for two decades, and the cross-continent round trip is set by the speed of light. Colin Scott’s interactive version tracks the drift year by year.

What the network adds

Four components show up on every hop:

• Propagation — ~5 µs per km one-way in fiber (light travels at about two-thirds c), so ~10 µs per km round trip. This is a hard floor: New York to London is ≈5,600 km ≈ 56 ms round-trip minimum, ~70–80 ms in practice. No amount of tuning beats it — the point Steve Souders made for coast-to-coast US back in 2010.
• Serialization — bits divided by link rate. A 1,500-byte frame over 1 Gbps takes 12 µs to clock onto the wire.
• Switching and routing — ~1 µs for a cut-through switch, more store-and-forward; routers, load balancers, and firewalls each add tens of microseconds to low milliseconds, and worse under load.
• Queuing — the variable term, and exactly where Little’s law bites.

Delay to a customer

Sum the path (RTT is the round-trip time):

total ≈ last-mile RTT
      + Σ propagation (distance × ~5 µs/km, each way)
      + Σ per-hop device/processing
      + connection setup (n × RTT)
      + server compute
      + dependency calls (serial = Σ, parallel = max)

The last mile usually dominates: wired broadband ~5–30 ms, mobile ~30–50 ms, geostationary satellite ~600 ms, low-earth-orbit ~25–60 ms. The lesson falls out of the arithmetic: last-mile and intercontinental propagation swamp your server’s millisecond, so an edge or CDN near the user beats shaving server compute — and a serial dependency fan-out (the Σ term) is the silent killer you can often parallelize down to max.

Treating the system this way — numbers flowing through a graph of components — is informal dataflow reasoning; doing it rigorously across a whole architecture is a post of its own.

Throughput

Little’s law took the arrival rate λ as given; throughput is how high λ can go. A system runs only as fast as its slowest node, so the recipe is to find that node, then move it with one of two levers.

Scale out. Throughput adds across parallel instances — double the slowest tier and you double its ceiling, until the next tier becomes the bottleneck. The slowest node, not the average, sets the budget.

Shed load before it arrives. A cache changes how many requests reach the slow tier at all. With a 90/10 hit split, only a tenth of traffic touches a database that tops out at 1,000 QPS, so the whole system sustains 1,000 / 0.1 = 10,000 QPS. The same split cuts expected latency the same way — 0.9 × 1 ms + 0.1 × 100 ms ≈ 11 ms — but that 11 ms is a mean, fit for capacity math and little else: nobody is actually served in 11 ms, and the unlucky tenth still waits the full 100 ms. The cheapest query is the one you never make.

Two limits keep the napkin honest:

• A ceiling per connection. A single TCP stream can’t exceed RWIN / RTT — the receive window over the round trip. A 64 KB window — the classic figure from before window scaling — across a 100 ms link caps one stream at ~640 KB/s however fat the pipe; modern stacks autotune windows into the megabytes, which raises the ceiling without removing it. Long-haul bulk transfer is window-bound, not bandwidth-bound.
• Average is not peak. Nodes slow under load — GC pauses, lock contention, cache eviction — so the sustained rate sits below the benchmarked peak. Size for the rate you’ll actually run at and keep a margin above it for the spikes — that margin is the headroom from the opening list, picked on purpose rather than discovered in an outage.

Failure rates multiply

Availability composes by multiplication. A request that depends on N services, each up 99.9% of the time, succeeds only 0.999^N of the time — ten such dependencies already drag you to ~99%. Each dependency you add is another factor below one and another latency floor.

Make the nines concrete first. A failure rate is 1 − availability, and a year of it is (1 − availability) × 365 × 24 h:

Availability	Downtime/year
99.999% (“five nines”)	5.3 min
99.99%	53 min
99.9%	8.8 h
99%	3.7 days

To compose them on a napkin, skip the multiplication: when failure rates are small, add them. (1 − n)(1 − m) = 1 − n − m + nm ≈ 1 − n − m, since nm is negligible. Three dependencies at 99.9%, 99.8%, and 99.7% sum to a failure rate of 0.001 + 0.002 + 0.003 = 0.006 — about 99.4% together, exact to four digits (99.4011%).

The sign flips with redundancy. Serial dependencies multiply availabilities, so each one you need shrinks the product. Redundant copies multiply failure rates instead: three independent 99.9% replicas fail together only 0.001^3 of the time — 99.9999999%. The word independent carries that figure: replicas that share a deploy, a bug, or a rack fail together far more often than the multiplication promises. Whether multiplication punishes or pays depends on whether the parts are all-required or any-suffices.

The nine nines hide a second assumption: any-suffices holds only while the survivors can carry the load. Say each of three replicas handles 100 req/s and the sustained load is 210 req/s. Together they offer 300 req/s — comfortable. Lose one, and the remaining 200 req/s sits below the arrival rate: the system is no longer stable, the queue grows without bound, and the spiral we met sizing pools finishes the job. No triple failure required — one dead node takes the other two down with it. Redundancy pays only when the nodes we keep can absorb the load of the nodes we lose; otherwise the balance is far more delicate than “all three failed — highly unlikely”.

The downtime table hides one too: those 8.8 hours a year are not sprinkled evenly across the calendar. If the business has a well-defined peak — Christmas prime time for a retailer — guess when the system will pick to go down. (Ask me how I know.) Nothing mystical: peak time is when the system is tried hardest — the most requests, the most complex ones, the least headroom — so failure probability concentrates exactly where an hour of downtime costs the most. For a system with known peaks, plan ahead and add capacity before the peak, just to be sure — especially when past data gives a forecast to size against.

Then prototype what it rests on

An estimate is a filter, not a verdict: it rules out the designs that can’t work and narrows the field for almost nothing. But the few pieces the whole architecture leans on earn a second pass — prototype them under a realistic workload before committing to the build. The napkin says a fixed pool should sustain 250 req/s; a load test on representative traffic tells you whether it does, where the knee is, and which term you guessed wrong. Estimate everywhere, measure what’s load-bearing, then build.

Summary

Four formulas and one ladder decide most of an architecture: L = λW for capacity, propagation for the latency floor, the path sum for delay to a customer, multiplied availabilities for failure, and the orders-of-magnitude ladder for everything else. The estimating itself needs no profiler, prototype, or benchmark — just arithmetic, done first. The estimate that rules out a design on a napkin is the cheapest one you will ever make.