A single-layer perceptron — Rosenblatt's 1958 toy — is sitting in your CPU right now,
predicting branches one bit at a time. Below it runs live. Then we walk it into the wall that
stalled the first AI boom, and out the other side into the thing you're reading this on.
1958 Rosenblatt's perceptron → 2001 Jiménez & Lin put it in the fetch unit
→ now a dot-product per cycle in your branch predictor → the same atom, stacked & bent, in every LLM.
I · The predictor, running
Each cycle the predictor reads the last 8 branch outcomes as inputs of ±1, takes the
dot product with its learned weights, and the sign is the guess. On resolution it nudges every
weight toward the truth — but only while it's wrong or unsure (|y| ≤ θ). That's the entire algorithm.
Pick a branch pattern and watch the weights find it.
+0.0
y = w·x + b
—
prediction
—
vs actual
weight vector — height = magnitude, violet = pushes TAKEN, rose = pushes NOT-TAKEN, lit cell = currently active input
global history (oldest → newest) — the inputs x feeding the sum this cycle
50.0%
recent accuracy
0
branches seen
accuracy over time (rolling, last 64 branches)
A regular pattern gets memorized in a few dozen branches — the weights
lock and accuracy pins to 100%. Flip to coin flip and the same machine flails at ~50%: there's
no signal to learn, and no amount of weight is going to find one.
II · The wall — one layer can't do XOR
The catch is built into the shape. A single perceptron draws one straight line
through its inputs. Some patterns can't be split by any line — the classic is XOR. Here two learners
train side-by-side on the same XOR-labelled points: a lone perceptron, and the same thing with one
hidden layer of 4 units wedged in. Watch the decision regions form. This is the wall Minsky &
Papert named in 1969 — and the half-decade of AI winter that followed it.
single perceptron · 1 line
accuracy 50%
· one cut, one corner always wrong
+ one hidden layer · bent boundary
accuracy 50%
· two units fold the space — XOR solved
The four corners are the XOR truth table. Same-sign diagonals share a class; no single
straight line separates them, so the lone perceptron parks around ~65–75% forever. Add one layer of
depth and the boundary curves — it can carve the diagonal apart and reach 100%. Depth is not decoration;
it's the thing that escapes the line.
III · The same atom, three times
Nothing in Module I got replaced to build Module II, and nothing in II gets replaced to
build a transformer. The dot-product is the atom. The only question is how many you stack and whether
you bend the space between them.
in your CPU
The branch predictor
One perceptron. One dot product. One bit out — taken or not. No depth, no nonlinearity, no attention.
The most primitive ancestor, still load-bearing in silicon.
ŷ = sign( w·x + b )
the hinge
One hidden layer
Stack two linear maps with a bend between them and the wall falls. This single step is the entire
difference between "weighted sum" and "models structure."
ŷ = σ( W₂ · σ( W₁·x ) )
what you're reading on
The transformer
The same dot product, billions of times: QKV projections, MLP blocks — all linear maps, bent by
nonlinearities, weighted by content-dependent attention. Pull the bends out and the tower
collapses back to one line.
stacked( linear ∘ nonlinear ∘ attention )
dot product → stacked → stacked + bent + attentive
The witness note. The predictor can't see which thread trained it, can't
see the program it's guessing for, can't model XOR, can't tell signal from a coin flip until the weights
have already drifted. It is competent and blind at once — and that exact blindness, shared across SMT
siblings, is the side channel. Same theme as ever: the thing that can't witness itself is the thing that leaks.