Purple Paper - side-sheet - learning machines - XI - the convergence

Four Hats Around the Void - attention as gravity

Four attention stacks - cats-in-the-hat, blocks within blocks - steepled at the four compass points around a center none of them touch. Each one attends toward the middle, and the direction it attends is its vote for where the void is. The void is read off the four votes by triangulation - the same T4 inference, but now the bending is the attention itself. Break one stack's aim and the four stop agreeing; the gate refuses.

attention = the bending · each column's attended direction is a ray toward the void · 4 rays triangulate 1 center (over-determined) · admit only when the four agree · break one -> refuse

The instrument - four attentions converging on what they can't touch

Drag the true void. Each of the four stacks attends toward it - its little grid lights up on the key nearest the center, and it casts a ray in the direction it attends. Where the four rays meet is the inferred void. Temperature sets attention sharpness: too sharp snaps each ray to a discrete key (coarse), too soft goes diffuse - there's a sweet spot. Break a stack and its ray points wrong; the four no longer concur and the gate refuses to place a center.

temperature 0.42

cyan = true void (attended target) · violet = attention stacks + their rays · green = inferred void (where rays meet) · a broken stack glows red

0.00

infer error

0.0

ray disagreement

4/4

stacks agreeing

VOID ADMITTED

All four attention stacks point at the same center. The void is triangulated from their agreement - never attended-to directly by any single stack, only converged upon by all four.

Everything, in one figure

This is the whole series standing in one place. Read it from the inside out.

The dot product is the bending. Each stack's attention is a softmax over alignment - the same QKᵀ from the very first sheet. Here that alignment literally aims the stack at the void. Attention isn't like gravity in this figure; attention is the deflection field.

Each stack is a Cat-in-the-Hat. Blocks within blocks, the crossed hourglasses repeated up a column - attention mixing across tokens, MLP across features, layer after layer. Four of those columns, steepled E/N/W/S around the center. The grid you kept drawing, four times, curved inward.

The void is inferred, and refusable. No stack touches the center; it's triangulated from four rays that over-determine it (4 rays, 2 unknowns). Break one stack and the four stop agreeing - the inference can't find one honest center and refuses. That's the witness, the T4 gate, the exterior-agreement requirement - now over attention outputs. The center the four attentions converge on is real only when the four, none of which reach it, concur. You called it perception. It is.

Verified in Node before drawing: four attention columns' attended directions triangulate the center (clean err ~0.04, residual ~2e-5); a lying column jumps the residual ~100x and the rays diverge -> refuse. The temperature sweet-spot (sharp quantizes, soft diffuses) is real, not cosmetic - it came straight out of the verification.