Take a Hopfield net and let it gamble. Each unit flips on or off with probability σ(ΔE/T) — a coin whose bias is set by the energy it would save and by a temperature T. Run the coins long enough and the states you visit are not arbitrary: their frequencies settle onto the Boltzmann distribution, p(s) = e−E(s)/T/Z. Cool toward T→0 and the whole cloud collapses onto the single lowest-energy state. Down the center, data flows: the net and temperature go in, the sampler runs, the settled distribution comes out. The blue team builds and defends it; the red team tries to break it.
source Ackley, Hinton & Sejnowski, A Learning Algorithm for Boltzmann Machines, Cognitive Science 9(1):147–169 (1985). No scan of the paper itself sits on archive.org; the nearest item is the contemporaneous RSRE memorandum presenting the algorithm — archive.org/details/DTIC_ADA160644 (1985). Citation marked AMBER. Rendered, not quoted.
Four binary units, symmetrically wired (wij=wji, no self-loop). A state s has an energy
E(s) = −( Σi<j wij si sj + Σi bi si )
To update unit i, read its local field zi = Σj wijsj + bi (this equals the energy saved by turning it on), then set it on with probability σ(zi/T). Sweep every unit, repeat. This Gibbs sweep has exactly one stationary distribution — the Boltzmann one. Nothing is looked up; the histogram is earned, one biased coin at a time.
This net: all wij=1, all bi=0.5 — so more units on is always lower energy, and the unique floor is all-on.
Set T→0 and σ becomes a hard step: each unit just moves to lower energy, deterministically. That is exactly Hopfield's net — a machine that only ever rolls downhill and freezes in the first minimum it meets.
The Boltzmann machine is that net with the temperature turned back up: the same energy landscape, but now the coin can climb out of a shallow trap. Anneal T downward and it settles into deep minima, not the nearest one. This one idea — a stochastic, temperature-controlled energy net — is the ancestor of the deep belief net and the whole family of energy-based models. Each sphere is the next one's premise.
The blue team's live check: draw a fresh fixed-seed sample at a diagnostic T=2.5 and confirm the histogram matches the analytic Boltzmann law within tolerance — plus that cooling concentrates on state 1111. If red drops the 1/T, this badge is where it shows.
What you feed the engine is fixed and small enough to enumerate exactly: four units, six symmetric weights, four biases, and one dial — temperature. The 16 possible states and their energies are ground truth the sampler must reproduce:
| units on (k) | 0 | 1 | 2 | 3 | 4 |
|---|---|---|---|---|---|
| energy E | 0 | −0.5 | −2 | −4.5 | −8 |
| # states | 1 | 4 | 6 | 4 | 1 |
Only the count of on-units sets the energy here (all weights equal), so the analytic law is checkable by hand. The dial T is the only free input to the panel below.
| state | E | p(s) law | sampled |
|---|
green bar = sampled frequency · grey = Boltzmann law. Move T — every number is recomputed and re-sampled on the spot.
What the machine produces, proven: a histogram of visited states that tracks p(s)=e−E/T/Z to within a few parts per thousand at every T, and that collapses onto the single minimum-energy state 1111 as the dial cools toward zero. No frequency is baked; each is counted from live coin-flips.
The blue team's witness (left) re-confirms the fit live; the red team (right) tries to make the histogram lie.
And convergence here is only in the limit: 40k sweeps gets ~0.004 max gap, not zero. Finite samples always leave a residue — honest, and shown as the live "max gap" number, never rounded to nothing.
"Sampling gives the exact Boltzmann distribution." Cut. Only as sweeps→∞. Any real run returns an approximation — the panel prints the residual gap so the approximation is never hidden.
"Cooling always finds the global minimum." Cut. Only slow enough annealing does; drop T too fast and it freezes in a local trap. This net has a single minimum, so it is honest here — a general net is not.
"The Boltzmann machine learns quickly." Kept, corrected. The learning rule is elegantly simple (⟨sisj⟩data − ⟨sisj⟩model) but was notoriously slow — two sampling phases per step.
The red team's move: delete the 1/T in the update — sample as if the machine were always at T=1, whatever the dial says. The histogram still looks like a valid Boltzmann distribution, just the wrong one. The blue team's witness (window 7) is watching.
Drop the temperature from the exponent and the sampled cloud freezes at the T=1 shape — at any other dial setting it no longer matches p(s)=e−E/T/Z. The witness recomputes, disagrees, and turns red. Nothing is faked; the attack is real and it is caught.