Purple Paper · side-sheet · learning machines · VII · the close

The Attention Hourglass — two compilers, one prediction

Eleven layers, apex to apex. Two compilers descend — a query side and a key side — and meet at the waist, where they collapse to a single number: the dot product. That scalar is the prediction. The value runs the bypass around the pinch and is read back out, weighted by what the waist decided. It's attention, drawn as your base-11 hourglass — and it's where every thread in this series ties off.
Q & K = the two compilers · QKᵀ at the 0-waist = the dot product = the prediction · V = the residual bypass · drag the query and watch the alignment decide

The waist, live — drag your query, watch it align

Four key tokens sit around the space — call them yours. The draggable token is the query — mine. The waist scores each pair by dot product (how much they point the same way), softmax turns the scores into the prediction (which key wins), and the output is the value-weighted blend. Temperature is the same knob from before: low T sharpens the prediction onto one key, high T diffuses it across all four.

drag the green query · line brightness = attention weight · pink = the predicted (winning) key · amber = output
the 11-layer hourglass · Q,K compile down to the pinch (the prediction) · V bypasses · out reads up
attention weights (the prediction) · the tallest is argmax = where the query aligns
The pinch. Each query·key is two numbers in, one number out — the score. That single scalar is the 0-layer, the narrowest point, the IR-waist. Two d-dimensional vectors crushed to one alignment value. Everything routes through it.
Drag the query toward a key and the dot product with that key climbs, its weight dominates, the output snaps to that key's value — a hard prediction. Drag it to the center, equidistant, and the four scores tie: the prediction goes diffuse, the output collapses toward the origin (the average of everything). Crank temperature up and even an off-center query gives a diffuse prediction — the waist stops committing. That's the whole of attention: align, score, commit, read out.

Where every thread ties off

This last object isn't new — it's the others, seen at once. The base-11 hourglass is the frame; the dot product is the pinch; and each earlier paper is one feature of this one machine.

paper I

the dot product is the waist

The atom from the branch predictor is the attention score. QKᵀ is the same weighted sum, now deciding where to look. Alignment = prediction.

papers III / cube

V is the residual bypass

The value doesn't pass the pinch — it's carried around it, read out weighted by the prediction. The cube's body-diagonal, exactly: the signal that survives the gate.

your N+M principle

the waist is the IR

Two compilers meet at one shared representation instead of every source wiring to every target. M+N, not M×N. The pinch is the interface that makes the saving.

the temperature thread

T is prediction sharpness

The same softmax temperature: low T commits to one key (hard gate), high T diffuses across all (soft blend). The dial between selection and summation, on the prediction itself.

The base-11 reading. Five layers down (Q,K compiling), the 0-waist (the dot-product prediction), five layers up (V,out expanding) — eleven, symmetric about the pinch, gravity pulling the signal down to the prediction and cost rising to read it back out. Two compilers, mine and yours, meeting at a single scalar they both agree on. That scalar is the only place the whole structure is one number wide — which is exactly why it's the waist, the IR, the gate, and the prediction, all at once. You drew attention without naming it, from an hourglass.

The last curse. Attention is 2017. The dot product under it is 1840s. The hourglass-with-a-waist as the efficient interface is every protocol stack and compiler IR ever built. Softmax is Boltzmann, 1868. You stacked seven papers of century-old stones into the shape of the newest machine in the world — and the newest machine turned out to be the same stones. lol. The frontier isn't a new stone. It's seeing they were always one wall.