The perceptron could not draw a line through XOR — the-first-neuron hit that wall and stopped. This machine climbs over it. A 2-2-1 sigmoid network, its weights nudged by the chain rule, learns the function no single layer can: the error flows backward through a hidden layer, and depth crosses the wall. The blue team builds and defends the descent; the red team strips the hidden layer to prove the wall is real.
source Rumelhart, Hinton & Williams, Learning representations by back-propagating errors, Nature 323:533–536 (1986). The Nature letter is paywalled and not on archive.org; rendered from the companion technical report Learning Internal Representations by Error Propagation (ICS 8506, same three authors) — archive.org/details/DTIC_ADA164453. Rendered, not quoted.
Each pattern runs forward: hidden hj=σ(∑i W1jixi+b1j), output o=σ(∑j W2jhj+b2), with σ(z)=1/(1+e−z).
Then error runs backward. Output delta δo=(o−t)·o(1−o); hidden delta δj=δoW2j·hj(1−hj). Every weight moves down its own gradient: W ← W − η·∂E/∂W. That is the whole trick — the o(1−o) factor is the derivative the perceptron never took.
Live gradient of one weight, checked against a finite difference:
Minsky & Papert (1969) proved a single-layer perceptron cannot represent XOR — no straight line separates {0,1,1,0}. the-first-neuron stops exactly there.
This sphere is the answer: add one hidden layer and a way to train it. Backprop assigns blame to the hidden units, and two of them are enough to fold the plane so the classes separate. The previous sphere's wall is this sphere's premise — depth plus a gradient is the door.
The blue team's live check: train a fresh network from seed 7 under the current wiring and confirm it reaches the XOR truth table with loss below 0.02. If red strips the nonlinearity, this badge is where it shows.
The whole training set is four points — the XOR truth table. Two inputs in, one target out:
| A | B | target |
|---|---|---|
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
Architecture 2-2-1, sigmoid units. Weights init U(−1,1) from a fixed seed 7; learning rate 0.5; full-batch gradient descent, up to 20000 epochs. Fixed seed ⇒ reproducible ⇒ node-verifiable. Feed it to the panel below.
Loss on a log axis: it starts at 0.25 (a coin flip) and dives four orders of magnitude as the hidden units learn the fold.
| A | B | target | output | ok |
|---|
Every number is computed live from the weights — nothing is looked up.
What the machine produces, proven: from seed 7, after 20000 epochs the four outputs land at ≈{0.01, 0.99, 0.99, 0.01} — each within 0.1 of the XOR targets {0,1,1,0} — at final loss ≈0.00014 < 0.02. A network with no hidden nonlinearity cannot beat 0.25. The wall is crossed, and it is measured.
The blue team's witness (left) retrains and confirms this live; the red team (right) strips the layer to break it.
And it is fragile: with only two hidden units, some initializations fall into local minima and never reach XOR. This sphere hard-codes seed 7 precisely because success is init-dependent — that dependence is disclosed, not hidden. The method is a triumph of engineering, and honest about what it is not.
"Rumelhart, Hinton & Williams invented backpropagation." Cut. Reverse-mode differentiation is older — Linnainmaa (1970), Werbos (1974) applied it to networks. Their 1986 letter named it, popularized it, and showed it learns useful hidden representations.
"Backprop solves XOR from any starting weights." Cut. With 2 hidden units it can stall in local minima; convergence depends on init. Here seed 7 is fixed and stated — the panel is honest about the one that works.
"A single layer just needs more training for XOR." Kept, corrected. No amount of training helps — that is Minsky & Papert's theorem, and the TAMPER proves it: strip the hidden nonlinearity and loss sits at 0.25 forever.
The red team's move: remove the hidden nonlinearity — make hj linear (identity), collapsing 2-2-1 into a linear model. Then retrain and try to learn XOR anyway. The blue team's witness (window 7) is watching.
A linear model can only draw a line — XOR is not linearly separable, so the loss cannot fall below 0.25. This is the disclosed planted void: it is real, it genuinely changes the computed loss, and the witness catches it and turns red. This is the wall the-first-neuron could not pass.