“Premature optimization is the root of all evil” — our field’s favorite half-sentence, quoted far more often than the sentence it was cut from. Usually it serves as permission: build it fast, profile later, fix a thing or two, done. Let’s put the sentence back together and ask what it licenses: when is optimization premature, and when is “premature” the excuse?

This series has been about small wins: flatter control flow , tidier conditions , invariants , the jumps worth keeping . This one steps back to the meta-question hanging over all of them: which of these small wins are worth chasing at all? My answer, up front: more of them than the mantra allows. “Avoid premature optimization” is, in my experience, frequently wrong: it saves effort and cost now, and both get billed later. Skipping optimization wholesale and waiting for the profiler is a bet that a Pareto jump will be there to collect. The jump is rarer than the mantra assumes, and it does not recurse; what is left either way is a long tail left uncurtailed while the code was written, and by then the tail is structural.

The whole quote

The line is usually fired as a conversation-stopper. Here is what Donald Knuth wrote in 1974, in Structured Programming with go to Statements :

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail.

Two numbers carry the argument: 97% and 3%. Knuth’s split is an estimate — “say about 97%” — but its shape is the Pareto principle : a vital few parts of a program dominate its cost, and the rest barely move the needle. The sentences nobody quotes say how to tell the two apart: measure, because guessing which parts are critical fails. Read whole, the quote says: find the 3% by measuring, and forget the small efficiencies elsewhere. The popular reading drops the word small.

So the interesting questions are five: how do we know the 3% exists? what happens after we spend it? what keeps the tail short in the first place? how far down can it take us? and what does spending it cost?

Pareto, or the vital few

The optimistic case. We profile, one hotspot lights up, we fix it, and a chart drops by an order of magnitude. This is the Pareto jump: deep analysis, an “a-ha!” moment, then a heroic refactor of whatever surfaced. It feels great because it is rare and dramatic.

I met the pure form of it once. Contracting through a consultancy, I was sent to a large social network to find out why their main page took about forty-five seconds to come up cold, and the account page about ninety. Warm, both were fine, single-digit seconds, and since most visits were warm the problem had stayed a nuisance, not a fire. Those are the numbers I remember, not ones I can produce a trace for; the browser tools of the day were embryonic, and their own engineers had gone looking and come back empty.

So I worked by elimination. I snapshotted the page and looked at what it pulled: some HTML, a few JS files, a couple of stylesheets. I emptied the body and the page stayed slow. I removed the JS and it stayed slow. Then I sandwiched the stylesheet between two inline scripts, the first stamping a timestamp and the second subtracting it, and there it was.

CSS. In everything I had built until then, CSS was free. HTML could be slow, JS could be slow, CSS just worked.

The architecture explained how it got there. Building an application meant contributing its pieces: an HTML snippet, the code to generate it, some JS, a set of rules. The JS had an assembler. It worked out which applications a page carried and produced one file for that combination, so the next page with the same set was served the cached copy instead of a fresh build. The CSS had no assembler. A developer built an application, added its rules to the file, and the file grew. Every cold visit parsed all of it.

I never found the mechanism; I was off the project as soon as the problem was found. My guess: browsers of the day worked through the tags one after another, which is why the sandwich worked at all, and a cold visit paid for parsing a file that size while a warm one reused what the browser had kept with the cached copy. I never timed the download itself, so it may have been that. The brief was to find the reason; the fix belonged to them. I assume it was the obvious one, pointing the assembler the JS already had at the CSS as well. I tested the result later: cold start came down to single-digit seconds, where warm already sat. An order of magnitude, out of one file.

That is the Pareto jump in its pure form: an unsuspected component holding nearly all of the cost, and a remedy that becomes obvious once the measurement lands.

When the long tail wins

The Pareto jump has three uncomfortable properties.

It does not always recurse. Take the 80/20 label literally for a moment: the first jump removes 80% of the cost and leaves a fifth of what we started with. Can we re-apply Pareto to that remainder for another 80%, and again, and how many times?

Sometimes the head keeps having a head — diminishing but real. If the remainder is self-similar, the arithmetic is easy and discouraging:

Every round asks the same of us: profile, read, find the hotspot, refactor around it. Every round returns a fifth of the one before. By the third we hold 99.2% of everything the exercise could ever yield, and the fourth asks that same work again for six tenths of one percent. There is always another head. It stops paying long before it stops existing.

Sometimes the remainder is flat: no vital few left, just a uniform tail where every item costs the same and returns the same. Spread that last fifth over a hundred call sites and each one is worth two tenths of a percent; another ten points means fixing fifty of them. Effort scales with payoff, one for one. Then there is no shortcut, and the only path to more is grinding the whole tail.

The jump is often a symptom of bad design. The big win was available only because something was poorly built — a glaring hotspot waiting to be found. Well-designed code rarely offers that jump; its cost is spread across the long tail, and the tail is cured by hygiene: follow simple rules, skip the obvious inefficiencies, don’t pick an obviously bad algorithm.

But bad design is not always negligence. Shipping a product reveals usage patterns that no design could predict, and those patterns can invalidate a once-reasonable design. The Pareto jump is then a legitimate correction in light of new information, not cleanup after sloppiness.

Curtailing the tail

The tail accumulates while the code is written, and that is also when it is cheapest to trim. Two things keep it short, and neither is heroic.

The first is design, and only the design: the data flow, the round trips, what is computed once and what per request. Those should be optimal, or leave room for the improvement we can already foresee. Individual steps can stay as plain as they like. A design without that room is where the structural problems come from: the tail is spread through the architecture, and grinding it means rewriting it.

The second is habit: the trivial optimization, free to take at writing time and one of a hundred to take later. A recent example. One of my colleagues fetched his data like this:

const A = await getStuffA();
const B = await getStuffB();
const C = await getStuffC();

It does the job. Reviewing each other’s code, he noticed that in the same situations I write:

const [A, B, C] = await Promise.all([
  getStuffA(), getStuffB(), getStuffC()]);

Same values, three round trips in flight at once instead of one after another, about three times faster when they take the same time. I had not thought about it; habit had. It is not free of judgment either: when getStuffC needs something out of A, the dependencies have to be traced, and now and then the careful version comes out messier than a flat Promise.all(). But most cases are the trivial one — he was right, and a later pass over the code we already had picked up the low-hanging ones — and skipping the trivial ones on the strength of the mantra is how the tail grows, one call site at a time.

The floor

Even when a real 3% exists, optimization has a hard lower bound. Shave the airport overhead of a New York to London trip all we like; the flight itself is the floor, and no amount of 80/20 reasoning gets under a physical restriction.

Knowing where the floor sits is what tells us when further effort is premature — or simply pointless. A request already at its network round trip, a program already waiting on its disk: there is no 3% left to find.

My own first lesson in floors came from a compiler. An IBM/360 — later a /370 — arrived with PL/I F in the box, every time. The optimizing PL/I O was optional, and plenty of installations did not have it; getting it onto the machine in front of me was its own chore. O produced better code, compiled more slowly, and understood an extended syntax F could not parse. It was the better compiler by any measure anyone would print on a datasheet.

So naturally I used O. What programmer does not want the better language? I wrote against its extensions and enjoyed them.

It took me a while to notice that the half I could have defended, the better code, was buying nothing. My programs spent their time waiting on I/O, and no quality of generated instructions moves that number — the device set the floor, and my code was already sitting on it. What O did buy was problems: it was not on every machine I turned up at, and by then my sources needed it. So I scaled back to F. That meant giving up the extended syntax and the nicer facilities, and it was the right price to pay.

The order there matters, because it is the usual one. I did not ask where the time went and then choose the tool; I chose what I wanted, and the answer arrived later, as trouble. Foresight is not the transferable part of this story — I did not have any. The transferable part is being willing to go back, and backing out of an optimization is much harder than declining one. By then there is code leaning on it, and giving it up means giving up the parts we liked. The easier life turned out to be the smaller language.

What it costs

Suppose the win is there. It still has to be bought, and the price never appears on the chart that shows the improvement. So let’s list what we hand over, since we hand it over whether or not the 3% turns out to be real.

An optimization can encode an assumption with an expiry date. Long ago I worked with code that stored the year in a single character: '7' through '9' meant that decade, '0' through '6' the next. I knew the author, so I asked him about it. He was proud of it: they handled a lot of dates, and the saving was real when memory was counted in kilobytes. So I asked how he would write '7' in the following decade. Hardware was advancing so fast, he said, that in ten years we would have entirely different computers, different processors, possibly a different architecture. Nobody would remember our software. The hardware advanced about as he said it would. His program outlived the prediction anyway, and hit its own boundary before Y2K got a chance.

The obvious objection is that he should have used hexadecimal and got two digits out of the character, or the whole byte and got far more. Both are true, and both miss it: they accept his premise and argue about the implementation. The premise was the mistake, and it was the same mistake I made with O a section earlier: an assumption nobody checked. “Computers will be unrecognizable” is a claim about hardware. “Therefore my software will be gone” is a claim about software lifetime, and he never checked it — though it was the only thing holding the trade up. The rest follows from there: a private format that other tools could not read, and a saving in bytes, which were about to become abundant, paid for in correctness, which stays scarce.

The verdict expires too. An optimization is measured true on one machine, one runtime, one workload. Change any of them and the measurement may not survive. This series has the case in hand: Duff’s device beats a plain loop by a fifth on one Node release, by 3% four majors later, loses to it on a low-power CPU under one of those same releases, and wins by 40% on a different engine. The loop did not change; the ground moved under it. So an optimization is a standing obligation to re-measure, and code that is not re-measured does not stay neutral — it keeps all of its complexity and loses its reason.

It spends simplicity. The next section argues that simple code is usually the fast code as well. This is the same coin from the other side: nearly every optimization is a withdrawal from that account. An unrolled loop, a hand-packed record, a cache with its own invalidation rules — each buys speed with something we already had, which is code we could read in one pass and reason about without ceremony. The most expensive thing an optimization can take from us is the ability to argue the code correct.

It makes the code harder to move. Whatever we optimized becomes a thing the rest of the code depends on. Later requirements route around it instead of through it, because changing it means re-deriving why it was fast in the first place, and there is no time for that during a feature. A local decision stops being local.

Sometimes the trap is the thing that made the code fast. In that compiler’s era a program travelled as source — a tape of JCL and PL/I, compiled on whatever machine was in front of me, because binaries rarely survived the trip — so writing for O’s extensions made a compiler somebody else had to have installed a precondition for my own code running. The fast path and the portable path were not the same path, and I found out from the wrong side of it.

All of this is a case for rationing: the critical 3% is worth every item on this list; heroics spent in the other 97% cost exactly the same and get nothing back. That is Knuth’s sentence read as an economic claim.

Simple code is the common thread

If the long tail is cured by hygiene and the floor caps the jump, the constructive advice is almost embarrassingly plain: prefer simple code. Not only for maintenance, though that alone would justify it. Simple code is easier to argue correct and easier to argue fast — and it is rarely slower in practice: the hygiene above already rules out the bad algorithm and the serial round trip, and what remains, plain loops walking memory in order, is what caches and compilers are built for. The hygiene that keeps the long tail flat and the simplicity that makes a correctness argument cheap are the same discipline.

Summary

Most of this series preaches a long-tail discipline: no single technique buys much; the habit of applying all of them is what keeps the mess from compounding. Optimization is the one face where that is not quite true — a genuine vital-few win sometimes exists, exactly as Knuth’s “critical 3%” promised.

But even there the discipline reasserts itself. The jump does not recurse forever, it is bounded by a floor, and it is often a symptom of design debt taken on earlier. Read as a license to skip optimization until a profiler complains, the mantra is a bet that a jump will be there to collect; the bet usually pays out as a long tail left uncurtailed, and the bill is structural. That is why the mantra is so often wrong in my experience: it saves effort and cost now, and both come back as the tail. So the honest reading of the famous half-quote is the whole quote: forget the small efficiencies in the 97%, the ones that cost work. Keep the tail short while the code is written, with a design that is optimal or has room to improve, and with the trivial optimizations, the ones that cost only a habit, taken every time. Spend the heroics only on the real 3%, and only as far as the floor allows.

Coding tactics posts

The series so far, in order:

• Code linearization — flatten the control flow.
• Logical optimizations — reshape the conditions.
• Boolean algebra — the algebra underneath them.
• TDD as religion — why a green suite is not a proof.
• Loop and if invariants — what holds at every line.
• break/continue is the new goto — the jumps worth keeping.
• Exotic goto: generators and exceptions — the jumps in disguise.
• “Premature” optimization — this post.

One more is on the way: the series collected in one place.