Cells that fire together, wire together. In 1949 Donald Hebb wrote the first learning rule: when two units are active at the same moment, strengthen the wire between them by Δwij = η·xi·xj. Present a pattern a few times and the co-active pairs grow, the never-together pairs stay at zero — and the weight matrix becomes an association you can recall from a fragment. Down the center, data flows: a pattern goes in, the rule updates the weights, the association comes out. Blue builds it; red tries to break it.
source D. O. Hebb, The Organization of Behavior: A Neuropsychological Theory (Wiley, 1949) — archive.org/details/organizationofbe0000hebb. Rendered, not quoted.
The whole mechanism is one line applied to every pair of units:
R1 for each pair (i,j), add η·xi·xj to wij. With binary {0,1} units, that is η only when both fire. R2 the matrix is symmetric — wij=wji. R3 no self-connection — the diagonal stays 0. R4 repeat a pattern and co-active weights accumulate; a pair never active together stays exactly 0.
Live, for the presented pattern:
| pair | co-active? | weight now |
|---|
McCulloch & Pitts (1943) built the neuron as a logic gate — a fixed threshold over fixed weights. It could compute, but it could not change: the weights were set by hand and never moved.
Hebb, six years later, gave those same weights a way to move on their own — the first rule by which experience edits the wiring. That is the hinge of this arc: the neuron gains memory. The very matrix this rule builds is the storage rule the next sphere reads back as content-addressable memory. Each sphere is the next one's premise.
The blue team's live check: train a fixed pattern from scratch and confirm the co-active weight grew, the never-together weight is exactly 0, the matrix is symmetric, and a one-unit cue recalls the whole. If red flips the sign, this badge is where it shows.
Six units, u0…u5. A pattern is a binary vector: each unit is firing (1) or silent (0). You feed a pattern in, over and over, exactly as a stimulus recurs in time.
| symbol | means | in the rule |
|---|---|---|
| xi=1 | unit i fires | contributes η |
| xi=0 | unit i silent | contributes 0 |
| η | learning rate | step size per event |
| wij | wire i↔j | what the rule edits |
Only co-firing pairs (both = 1) ever move a weight. That single fact — feed it into the panel below — is the entire engine.
Weight matrix wij (symmetric, zero diagonal) — pink = strengthened, dark = still zero:
Recall from a fragment — cue only the first firing unit, clamp it on, let the wires vote:
Every weight is added live by η·xi·xj — nothing here is looked up or baked.
What the machine produces, proven: a weight matrix where every co-active pair of the stored pattern has grown to k·η after k presentations, every never-together pair sits at exactly 0, and the matrix stays symmetric with a zero diagonal. That matrix is a content-addressable memory: a one-unit cue reconstructs the full pattern. The witness (left) confirms it live; the red team (right) tries to make it lie.
Approximation flagged honestly: pure Hebb has no bound and no decay — the adversary (right) shows why that is a real defect, not a detail.
And the slogan hides the biology: real synapses obey timing (STDP, Bi & Poo 1998) — pre-before-post strengthens, the reverse weakens. Mere co-activation is a first approximation, not the law. Correlation is not causation here either: two features that merely co-occur will wire together whether or not one drives the other.
"Hebb wrote 'cells that fire together wire together.'" Cut. He did not. That mnemonic is a later paraphrase (Löwel & Singer, 1992; popularized by Carla Shatz). Hebb's actual clause is that cell A "repeatedly or persistently takes part in firing" B — Organization of Behavior, 1949.
"The Hebb rule is how the brain learns." Cut. It is one mechanism. Inhibition, neuromodulation and spike-timing all shape real plasticity; unmodified Hebb is not biologically sufficient.
"Unbounded weight growth is harmless." Kept, corrected. Runaway potentiation is a genuine failure mode — the whole family of Oja / covariance / BCM rules exists to fix it.
The red team's move: flip the sign of the update to −η·xi·xj (anti-Hebbian). Now co-firing pairs decrease instead of grow — the wire the pattern should build gets destroyed. The blue team's witness (window 7) is watching.
Flip the sign and the co-active weight goes negative, the one-unit cue no longer recalls the pattern, and the witness recomputes, disagrees, and turns red. Nothing is faked; the attack is real and it is caught.