Read a transformer's running guess by projecting the residual stream through the unembedding early. Apply the final LayerNorm and W_U to the residual at every layer, not just the last, and watch the prediction sharpen with depth. Rendered, not quoted.
source nostalgebraist, interpreting GPT: the logit lens (2020), LessWrong. No stable arXiv id — cited by post/year, marked AMBER.
A constructed transformer-like stack with known weights. A residual stream r of width d=4 is updated by three fixed layer contributions, then a final LayerNorm and an unembedding W_U (5×4) turn it into logits over a 5-token vocabulary.
r₀ = [0.8, 0.4, 0.3, 0.5] → +δ₁ +δ₂ +δ₃, each δ pushing mass onto token 2.
LN(x) = (x − mean) / √(var + ε), ε=1e-5, γ=1, β=0. Token 4's row is the flat vector ¼·1, so its logit is exactly 0 — a fixed baseline. Every readout is closed-form; no training.
the-unembedding, applied early. The lens is nothing but W_U — the map from residual space to vocabulary — reached for before the last layer. Reading the residual stream as a running prediction is the simplest window into what a transformer is thinking, layer by layer.
Neighbour sphere: the-unembedding (how the final residual becomes token logits). The logit lens borrows that same matrix and points it at every depth.
Live re-check of the whole instrument at boot and on demand. Confirms the final-layer lens reproduces the true output logits, that depth sharpens the guess, and that the LayerNorm is not skipped.
witness — running…
The residual stream at each depth — a context already partly processed. Nothing is fed a token id; the lens reads the vector state directly.
r₀ … r₃ loading…
logitsℓ = W_U · LN(rℓ) at every layer ℓ. Bar = probability mass on the eventual top token (2). The guess is still forming early and sharpens with depth.
Top token per layer, with softmax confidence in token 2:
The proven result: the final-layer lens equals the model's actual output logits to within 1e-9, and token 2 has emerged as the prediction.
result — pending…
The lens is not a faithful decoder of every model. It works when intermediate layers already write in the output basis (residual stream directly readable by W_U). In many trained models the early-layer bases are rotated or scaled, so the raw logit lens produces garbage until a learned affine map (the "tuned lens") is fit per layer.
Here the wall is honoured by construction: this model's layers do write in the output basis, so the plain lens is faithful. That is a property of the toy, not a universal law.
"The logit lens shows the true belief of every intermediate layer."
↳ It shows what W_U would decode from that residual — an approximation. Faithfulness varies by model and layer; late layers are usually far more readable than early ones.
"You can drop the final LayerNorm; it barely matters."
↳ Skipping LN before W_U breaks the match to the real output (see window 6). LN sets the scale the unembedding was calibrated against.
The disclosed planted void: skip the final LayerNorm and feed the raw residual straight into W_U. The final-layer readout no longer matches the true output logits — the WITNESS in window 7 catches it live and flips red.
state: intact — LN applied.