A plumb-line tells you whether a wall is true — not whether it is tall. This one tests whether a model's confidence is true to its accuracy. The stream is overconfident; drop the plumb-line (temperature) until the wall reads straight.
LIT ECE / MCE & the temperature fit, computed live FIG a labelled toy logit field WALL a real bake under shift
SELF-CHECK running…
Reliability diagram — eval split
■ accuracy per bin ● mean confidence ▨ the gap · the dashed diagonal is the plumb-line (perfect: conf = acc). Bars below the line = overconfident.
Live readout — eval split
Negative log-likelihood vs T
Fit on the calibration split (NLL is convex in 1/T, the inverse temperature). The dot is the minimum; the dashed line is your current T. Accuracy does not appear here — T can't move it.