A forecast of ‘70% chance of rain’ is only honest if it rains about 70% of the times you say it. Plot claimed probability against what actually happened; a calibrated forecaster lands on the diagonal. Slide calibration (or tap) and watch the Brier score fall.
A reliability diagram and the calibration part of the Brier score. Ten probability bins (0.05…0.95); a perfectly calibrated forecaster's observed frequency equals its stated probability (on the diagonal), an over-confident one pushes its outcomes toward 0 and 1. The slider blends between them; the calibration score Σ (fᵦ − oᵦ)² / 10 is the mean squared gap from the diagonal — zero when calibrated, large when not. A fail-loud self-check throws unless the score is ~0 at full calibration and strictly larger when over-confident.
The over-confident distortion is a fixed illustrative curve, and only the calibration (reliability) component of the Brier decomposition is shown (not resolution/uncertainty). The bins, gaps and mean-squared score are computed exactly.