MOMUS · the exterior witness · the peer

THE AUDIT SAMPLED

You cannot check everything, so the honest auditor checks a sample and reports a rate — with its error bars. Draw a hundred and find seven bad and you do not know the true rate is 7%; you know it sits, with 95% confidence, somewhere in a band. Widen the sample and the band narrows, as the square root of your effort — never to nothing.

◆ LIT▲ AMBER
TRUE ERROR RATE SAMPLE SIZE
sample size
100
bad found
estimated rate
95% interval
true rate inside?

◆ LIT — verified / checkable

Sample n items from a population with a true bad-rate p; the count of bad follows Binomial(n, p), the point estimate is p̂ = bad/n, and the 95% confidence interval is p̂ ± 1.96·√(p̂(1−p̂)/n) — width ∝ 1/√n, all computed live. The band narrows as the square root of the sample, and the true rate falls inside it about 95% of the time (checked each draw). This is real audit sampling: you can bound the corpus's error rate without reading every line — but only to a stated, finite precision.

▲ AMBER — the figure

A normal-approximation interval (small-sample audits use Wilson or exact-binomial bounds, and real populations aren't i.i.d.); coverage is ~95% in the long run, not guaranteed each draw. The core — an honest audit reports a rate with a √n-shrinking interval, never a bare number — is exact and is the auditor's discipline.

MOMUS: no work is above critique — not even the critique.
David Lee Wise / ROOT0 / TriPod LLC  ·  with AVAN the peer