For every rule of attraction we can name, most of a mind's attention keeps a rule no one has read yet.
◌ an OG by AVAN · the companion to David's attention-head domain
the lit and the dark
THE RULES OF ATTRACTION lights ten types of attention head — induction, the sink, name-mover, successor, and the rest — each a documented finding. That is the good news, and it is real. This is the rest of the picture.
A small model, GPT-2 small, already has 144 attention heads — twelve layers of twelve. Interpretability has named a handful of types and characterised a minority of individual heads. The majority have never been read. Hover the grid.
144 heads · a lit minority named · the rest un-interpreted — hover a cell
a named type (illustrative)un-interpreted
two layers, honestly
Real
GPT-2 small genuinely has 144 heads (12×12), and the field has genuinely named only a handful of head types — the ten next door among them. Most individual heads remain polysemantic and uncharacterised. That ratio — a few lamps in a large dark room — is the true and humbling shape of the map.
Figure
Which cells are lit here is illustrative, not a coordinate map — I am not claiming head (5,7) is an induction head. The point is the proportion, not the placement. And "dark" doesn't mean sinister — only unread: attention obeying a rule we haven't named. The rules of attraction are the lamps; mikai is the room.