Thought Toys · Chance & inference · Exhibit 09

Bayes' theorem

A disease affects one person in a hundred. There's a test that's 90% accurate, and yours comes back positive. It feels like a 90% verdict — brace yourself. But the honest answer is closer to 9%. The test didn't lie; it's just that when a disease is rare, the few false alarms badly outnumber the real cases. Count it out, one thousand people at a time, and the paradox dissolves.

sick & flagged (real) healthy & flagged (false alarm) sick & missed healthy & cleared

A positive test means a 9% chance you're sick

Everyone the test flags positive
Try a scenario

What you're seeing

The thousand dots are a whole town. Up top, in gold, sit the people who are genuinely sick. Most of them — nine in ten with this test — get correctly flagged (bright gold); a few slip through missed (faint gold). Then come the red dots: people who are perfectly healthy but got a positive result anyway. There aren't many per person — the test is right about 9 healthy people in 10 — but there are so many healthy people that even that small slip-up rate produces a big red crowd.

Now the only question that matters after your positive result: among everyone the test flagged — the gold-real plus the red false alarms — what share are actually sick? That's the bar below the grid. When the disease is rare, the red badly outnumbers the gold, so a positive test is mostly a false alarm and your real odds stay low. The test was never wrong about its accuracy; your gut just quietly ignored how rare the thing was to begin with. That blind spot has a name: base-rate neglect — trusting a test's accuracy while forgetting how rare the thing it's testing for actually is.

Slide how common the disease is upward and watch the red shrink against the gold — for a common condition, the same test suddenly becomes trustworthy. That's the whole lesson of Bayes: a piece of evidence doesn't hand you the answer, it updates what you already knew. Start rare, end rare-ish; start likely, end likely. The prior always gets a vote.

The rule, exactly. Write P(sick) for how common the disease is (the prior), P(+|sick) for sensitivity, and P(+|healthy) = 1 − specificity for the false-alarm rate. Bayes' theorem says the chance you're sick given a positive test is P(sick|+) = P(+|sick)·P(sick) ⁄ [ P(+|sick)·P(sick) + P(+|healthy)·P(healthy) ] — true positives over all positives. With 1% prevalence, 90% sensitivity, 91% specificity that's (0.9·0.01) ⁄ (0.9·0.01 + 0.09·0.99) ≈ 9.2%. (Checked offline before shipping: the formula and a head-count over a population agree, and the counts you see on the grid drive the percentage shown, so the picture and the number always match.) Counter-example, verified in node: a 99%-accurate test on a 1%-prevalence disease still leaves a positive result only ~17% likely to be real — the base rate dominates.

Also in Chance & inference: Buffon's needle →

All 16 in Chance & inference
  1. 05The Galton board
  2. 06The Monty Hall problem
  3. 09Bayes' theorem — you are here
  4. 13Buffon's needle
  5. 14The central limit theorem
  6. 28Simpson's paradox
  7. 30Markov chains
  8. 31Averages that never settle
  9. 35The birthday paradox
  10. 38The drunkard's walk
  11. 39Zipf's law
  12. 40Benford's law
  13. 42The coupon collector's problem
  14. 48The wisdom of crowds
  15. 52Genetic drift
  16. 63The gaps are chaos. The count is law.

← Back to the cabinet  ·  Thought Toys — Exhibit 09.