Given the data, which parameters make what we observed most probable? Maximum likelihood answers by climbing the log-likelihood surface to its peak. This instrument fits a Gaussian and a Bernoulli to fixed-seed data in closed form and proves the estimate is the exact stationary maximum. Rendered, not quoted.
source R. A. Fisher, On the Mathematical Foundations of Theoretical Statistics (1922), Phil. Trans. Roy. Soc. A 222, 309–368 · DOI 10.1098/rsta.1922.0009 — the paper that named likelihood.
Two parametric families, each a product of independent per-point densities:
Gaussian — each xi ~ N(μ, σ²). Log-likelihood ℓ(μ,σ²) = Σi [ −½log(2πσ²) − (xi−μ)²/(2σ²) ].
Bernoulli — each xi ∈ {0,1} with P(1)=p. Log-likelihood ℓ(p) = k·log p + (n−k)·log(1−p), k = Σxi.
Set ∂ℓ/∂θ = 0 and solve. The closed forms fall out exactly — no optimiser needed.
Minimising the-cross-entropy is maximising likelihood. Cross-entropy H(q,p) = −(1/n)Σ log pθ(xi) = −ℓ(θ)/n.
Down one is up the other. Every model trained by NLL / cross-entropy loss is an MLE machine: pick the parameters that best explain the data. Fisher, 1922, is the principle under each modern gradient step.
Live re-check: does the reported estimate still equal the sample mean, the exact MLE?
…
It re-derives μ̂ from the raw data on every render. If the panel (window 6) is tampered, this flips red immediately.
Fixed-seed samples (deterministic PRNG — mulberry32):
Truth is hidden from the estimator; it sees only the numbers.
Log-likelihood surface ℓ(μ,σ²) — brighter is higher. The + marks the computed peak (the MLE).
The proven estimate — the exact peak of the surface:
…
μ̂ equals the sample mean and σ̂² the mean squared deviation, to 1e−9, with zero gradient — a stationary maximum, not a guess.
"Most probable parameters" is not "true parameters." MLE maximises the likelihood of this sample — a different draw yields a different peak. With n=40 the estimate carries real sampling error; σ̂² is biased low by a factor (n−1)/n. Likelihood ranks parameters, it does not certify them.
"The MLE is unbiased." → False. The Gaussian variance MLE underestimates σ² systematically; only μ̂ is unbiased here.
"Higher likelihood always means a better model." → No. Push σ²→0 at a data point and ℓ→+∞ — unbounded; likelihood must be compared within a fixed family.
"MLE needs an iterative solver." → Not for Gaussian/Bernoulli — the maximum is closed-form and exact, computed here with no optimiser.
The disclosed planted void. This flips the engine to minimise the likelihood — it returns the worst-fit location (the point farthest from centre) instead of the mean.
The Witness (window 7) re-derives the true MLE from the data and catches the swap live.