INDUCTION HEAD

Match-then-copy. On a repeated sequence [A][B]…[A], the head looks back to the earlier [A], finds what followed it ([B]), and copies it forward as the prediction. The engine of in-context learning. Exact algorithm; sequence constructed to isolate it.
AMBER
2D · Quantitative Channel
CANVAS 2D
demo: sine — replace in draw2D()
CANVAS 2D UNAVAILABLE
3D · Spatial Channel
SOFT-GL · NO GPU · NO LIB
DRAG ROTATE · WHEEL ZOOM
demo: lattice + orbit — replace in build3D()/draw3D()
RENDER FAILED —
Notes

Induction head. The defining mechanism of in-context learning. On a repeated sequence, at each token it searches back for the previous occurrence of that same token and attends to whatever followed it — then copies that as the prediction. The heatmap shows the resulting off-diagonal induction stripe; the copy-score measures how often the copied token is actually the correct next token. Amber tier: the match-then-copy algorithm is exact, the sequence is constructed to isolate it (real induction heads need a model large enough to have formed them in training).