Before a model can use a repetition, a head has to NOTICE it. The duplicate-token head does exactly one thing: from any token, it looks back and attends to the earlier place the SAME token appeared. It’s the first move of the induction circuit — the memory that says ‘this has happened before, at position 12.’ Slide along the text and watch it find the echo.
For the token at position p, the duplicate-token head attends to the most recent earlier position j<p holding the identical token (or nowhere, if it is the first occurrence). The instrument runs a fixed sequence with repeats. A fail-loud self-check throws unless, at each repeated token, the head points to the correct previous index, and a first-occurrence token points nowhere — the exact ‘same-token’ lookup. Duplicate-token heads are a documented component of the induction circuit (Elhage et al. 2021; Olsson et al. 2022).
A demonstration of the lookup a duplicate-token head performs; a trained head does it via a key/query match on token identity, is approximate, and can attend to partial matches. The index lookup shown is exact.