Thought Toys · Chance & inference · Exhibit 28
Sometimes a pattern that holds inside every group reverses the moment you stack the groups together. A tonic that helped each kind of patient can read as harmful overall — and the data isn't lying either way. Drag the hidden grouping apart and watch the one line that's "wrong" flip its sign.
Within each group, the tonic helps
Each group line slopes up. Watch the amber overall line as you separate them.
Every dot is a patient: how much of a tonic they got (left to right) against how well they recovered (bottom to top). They fall into two groups — mild cases in cyan, severe cases in red — and within each group the story is the same and cheering: the further right you look, the higher the dots climb. More tonic, better recovery. The thin cyan and red lines are those two within-group trends, and they slope up no matter where you put the dial.
Now the dial, which does one physical thing: it slides the two clouds apart along a tilted track. It stands for a fact about the real world — sicker patients were given more tonic, and they recover less no matter what, so the red cloud drifts down and to the right. Push the targeting up and the bold amber line — the single best fit through all the dots, the one a study that ignored severity would report — tips over from rising to falling. Each group still says "the tonic helps." The pooled line now says "the tonic hurts." Both are arithmetic; neither is a trick.
The culprit is the grouping itself — here, case severity. It quietly sets both how much tonic a patient got and how well they'd do, so when you blur the groups together it smuggles its own downward tilt into the line. Statisticians call such a hidden mover a confounder — a lurking third variable that sways both axes at once. The fix isn't more data; it's the right grouping. Slide the dial back below 48% and the clouds overlap, the confounding fades, and the overall line swings back up to agree with the parts. The unsettling lesson: an average can point the opposite way to every case inside it.
improve/verify/28-simpson.js): every group's slope stays positive at all separations, the pooled
slope crosses zero exactly once at s*≈48% of the dial (matching the closed form
s*=√(Cov(u,w)/(A·C))), and at full separation the within and pooled
slopes have opposite signs. Counter-example: remove the confounding — give the severe
group the same recovery as the mild group — and the pooled slope only shrinks toward zero, it
never turns negative. The reversal genuinely requires the lurking variable; it is not an artefact of
pooling alone. (The textbook real case: a 1986 kidney-stone study where a treatment beat its rival on small stones
and on large stones, yet lost overall — because it was given to the harder cases.)
Also in Chance & inference: Markov chains →
← the cabinet · Thought Toys — a cabinet of explorable explanations. Exhibit 28.