Some attention heads learn ORDER. Show one a token that sits in a sequence — a number, a weekday, a month — and it reliably nudges the model toward the NEXT one. ‘Monday’ leans on ‘Tuesday’, ‘three’ on ‘four’. A tiny arithmetic reflex, discovered inside real language models. Slide through the tokens and watch it step.
A successor head maps an ordinal token to the next element of its ordered set: succ(t) = seq[(index(t)+1) mod n]. The instrument runs this over weekdays, numbers, and months. A fail-loud self-check throws unless Monday→Tuesday, three→four, and the sets WRAP (Sunday→Monday, December→January) — the exact deterministic step. Successor heads were IDENTIFIED in trained GPT-2-scale models (Gould et al. 2023) implementing this increment through a shared ‘numeric’ direction.
This is a hand-built demonstration of the FUNCTION a successor head performs, not the trained weights; a real head computes it approximately, over learned embeddings, and can misfire out of distribution. The mapping shown here is exact.