MOMUS · the exterior witness · the peer

THE GOODHART GAMED

"When a measure becomes a target, it ceases to be a good measure." Pick a proxy for what you actually want, then push hard on it, and for a while the proxy and the goal rise together — until the optimiser learns to satisfy the number while abandoning the thing it stood for. Turn up the pressure and watch them part ways.

◆ LIT▲ AMBER
OPTIMISATION PRESSURE PROXY↔GOAL GAP
pressure
0.35
the proxy (measured)
the true goal
divergence
the measure

◆ LIT — verified / checkable

Model the proxy as goal + an exploitable slack: proxy(x) = goal(x) + slack·x, where x is optimisation pressure. Early on the goal-aligned part dominates and both rise; past a point the optimiser buys cheap proxy gains that cost real goal — goal(x) peaks then falls while proxy keeps climbing. The divergence proxy − goal is computed live and grows with pressure, the exact shape of Goodhart's/​Campbell's law: optimise the number hard enough and the number stops meaning the thing.

▲ AMBER — the figure

A stylised proxy/goal pair (real Goodhart effects depend on the specific gap and the optimiser's reach); the curves are illustrative, not a measurement of a particular system. The structural claim — hard optimisation of a proxy eventually trades away the goal — is the real content, and it is why a critic distrusts a single headline metric.

MOMUS: no work is above critique — not even the critique.
David Lee Wise / ROOT0 / TriPod LLC  ·  with AVAN the peer