MOMUS · the peer · critique that earns its keep · kept by MOM

THE MULTIPLE COMPARISONS ◧ 2D · ◍ 3D · ◆ 4D · ◐ shadow · 👶 TAP

Test twenty true-null hypotheses at p<0.05 and you expect one ‘significant’ result from pure noise. Run enough tests and you can ‘discover’ anything. Slide (or tap) the correction from none to full Bonferroni and watch the false alarms go quiet.

◆ LIT▲ AMBER
◧ THE MEASURE · 2D
◍ THE FIELD · 3D · false alarms go dark
◆ THE FOURTH · 4D · a tesseract turns
◐ THE SHADOW · one dimension down
👶 THE TODDLER CORNER — tap to bring them together
tests
20
effective α
P(≥1 false alarm)
expected false alarms

◆ LIT — exact / checkable

With m independent true-null tests at threshold α, the chance of at least one false positive is the family-wise error rate FWER = 1 − (1−α)^m, and the expected count is m·α. At m=20, α=0.05 that is a 64% chance of a false discovery. The Bonferroni correction tests each at α/m, holding FWER ≤ α. The slider blends the threshold from α to α/m; a fail-loud self-check throws unless FWER falls from ≈0.64 to ≈0.05 across the sweep.

▲ AMBER — the figure

Independence of the m tests and a common α are the simplifying assumptions (real tests correlate; Bonferroni is then conservative, and Holm / Benjamini-Hochberg refine it); m=20 is illustrative. The 1−(1−α)^m and α/m arithmetic is exact.

MOMUS: praise is cheap; a fault named precisely is the gift.  — MOM
David Lee Wise / ROOT0 / TriPod LLC  ·  kept by MOM, with AVAN