Thought Toys · Computation & information · Exhibit 58
A checkout that's half busy barely has a queue. Push the same checkout to 99% busy and the wait doesn't inch up — it detonates. The line isn't made of load; it's made of randomness bumping into a server that's almost always busy.
The line now, and the wait vs. how busy the server is predicted wait live simulation
—
—
One server — a checkout, a help desk, a CPU core — handling jobs one at a time. Jobs arrive at random moments, and each takes a random amount of time to serve. Utilization ρ is simply the fraction of time the server is busy: arrivals per hour divided by how many it could finish per hour. You'd think the wait would track how busy the server is — 90% busy, a moderate wait; 99% busy, a slightly longer one. It doesn't. The wait tracks one divided by the idle fraction, and idle time is exactly what vanishes as you approach full: the average customer waits about one service time when the server is half busy, nine times that at 90% busy, and ninety-nine at 99%. Halve the remaining slack and you don't add a little wait — you roughly double it.
Drag utilization up and the amber curve barely lifts across the whole calm middle, then turns and rockets toward a wall at 100%. That wall is real: past ρ=1 the jobs arrive faster than they can be cleared, so no steady wait exists at all — the line just grows forever. The reason a busy line is so much worse than a half-busy one is randomness: when a burst of arrivals lands on a server that was already nearly always occupied, there's no idle gap to absorb it, so the backlog it creates takes a long time to drain. Flip to Clockwork — perfectly evenly spaced arrivals, perfectly equal service times — and at the same 99% utilization the line empties to nothing. Same load, no randomness, no wait. The line was never made of busyness. It was made of variability.
improve/verify/58-mm1-queue.js): a discrete-event simulation
streaming the same random arrivals and service times through one server has a running-average wait that
converges onto ρ⁄(1−ρ) at 50%, 80% and 90% busy; the closed forms satisfy
Little's law; and the wait is monotone and blows up hyperbolically (99% → 99 service times).
Negative control: make arrivals and service perfectly regular (a D/D/1 queue) and the
wait is exactly zero at 50%, 80% and 95% busy — so it is variability, not load, that makes the line;
and at exactly ρ=1 the simulated average never settles, it keeps climbing as more customers are
served (no steady state).
Also in Computation & information: The set that's only sure when it says no →
← the cabinet · Thought Toys — a cabinet of explorable explanations. Exhibit 58.