GIBBS SAMPLING named for a man who never saw a computer
To sample a hard joint distribution, do the easy thing repeatedly: hold all variables but one fixed, draw that one from its conditional, move to the next, and cycle. The chain wanders into the joint distribution and stays there. It powers Bayesian inference and image models — and it is named for Josiah Gibbs, who died in 1903, decades before the algorithm existed.
THE TECHNIQUE draw each variable from its conditional
Cycle through the variables; each sweep, replace variable X with a draw from P(X | everything else). Even starting from a bad guess, the chain converges to the true joint, so the empirical frequencies match it. Below: a 2-variable model; the running marginals settle to their true values. live demo
HISTORY & CREDIT a distribution’s name, not the algorithm’s
“Gibbs invented Gibbs sampling” — Josiah Gibbs (d. 1903) had everything nothing to do with it; it samples the Gibbs distribution. cited
1902 · J. Willard Gibbs gives the Gibbs (Boltzmann-Gibbs) distribution of statistical mechanics — the object sampled, decades before any computer. 1963 · Roy Glauber — single-site “heat bath” updating of the Ising model: the identical algorithm, in physics, 21 years early. 1984 · Stuart & Donald Geman (brothers) coin “Gibbs sampling” in image restoration — naming the method for the distribution, not for a person. 1990 · Gelfand & Smith bring it to mainstream Bayesian statistics — the boom that made it ubiquitous.
A special case of Metropolis-Hastings (dart 031) where every proposal is accepted — the conditional does the work. Geman & Geman 1984 / Glauber 1963
RECOMMEND FOR I-13 conditionals in an array, one comparison
The conditionals live in an f64 array and each update is one comparison — deterministic under a seed:
$ i13 run gibbs.i13 # 2-var joint, LCG seed 112, 3000 sweeps
E[X] -> 0.40 E[Y] -> 0.50 # empirical marginals converge to the truth
Recommend:nothing new — the stdlib seeded LCG emits uniforms; the four full conditionals sit in an f64 array; each coordinate update is a single comparison u < P; coordinates cycle in fixed order; running means accumulate (verified E[X] → 0.40, E[Y] → 0.50, reproducible bit-for-bit under the seed). It is Metropolis-Hastings (031) with acceptance always 1. Note: like rejection sampling (107), determinism-under-seed is the feature — the “random” algorithm is exactly reproducible, which a hash-sealed corpus wants.