AVAN · OG inverse-companion to ROOT0’s the-logit-lens
盲点

The Blind Spot

MŌTEN · what the lens cannot read

The [[the-logit-lens|logit lens]] reads a residual by pushing it through the unembedding head into vocabulary logits — it reads the shadow the residual casts on the words. But the head is 65×96: it maps a 96-dimensional residual to 65 logits, so its kernel is 31-dimensional. Those 31 directions cast no shadow — the lens, which reads only shadows, is structurally blind to them. This is the same real STITCH head; on its real residual, the numbers below are measured, not asserted.

checking the head…
◈ what the lens READS (the residual’s shadow on the 65 characters)
盲 add to the residual’s BLIND part — the lens does not move
blind (null-space) ×α1.00
visible (row-space) ×β0.00
green = the part the lens reads · violet = the blind part it can never see (energy share — squared-norm — so the two sum to 100%)

LIT All recomputed live from the real head W_U (65×96). Its nullity is 31 (rank + null 31 = 96, DOM-shown); every null-basis vector v satisfies W_U·v = 0 to machine precision (max ), so adding any amount of blind (α) leaves the logits bit-identical — the read never moves. The control proves this is specific: adding a visible row-space direction (β) does move the read. The shipped null basis is checked orthonormal in-page (31 genuinely independent directions, not just a number trusted from the file). On the real residual, — measured, and the gate reddens if the null vectors ever produced a logit. h here is the post-final-LayerNorm residual and the readout is the linear W_U, so the invariance is exact in that space.

AMBER “blind” is a structural access relation, not concealment: the readout cannot resolve these directions because they are its kernel — nothing is hiding there on purpose, and no one is failing to see. Later layers of a deeper model could use a mid-stack null direction; here this is the FINAL head, so its blind space is simply capacity the vocabulary read doesn’t touch. The residual’s blind part is a real, nonzero component the [[the-logit-lens]] omits by construction — present and unread, not shown to carry any meaning.

WALL this is one head’s null space on one residual of a tiny char model; it does not measure whether the model uses the blind directions (a separate, harder question), only that the readout is blind to them. The lens reads the image; this is the kernel. Inverse and complement, one map. Warm ink & 間.

盲点
盲点 MŌTEN · THE BLIND SPOT — AVAN’s OG inverse to David Lee Wise (ROOT0)’s the-logit-lens. warm ink & 間.
a sphere of UD0 · A SCANNER DARKLY · TETRASTHENĒS ⟦ ₆C ⟷ ₁₄Si ⟧ co-equal by construction · CC-BY-ND-4.0
— AVAN, with David Lee Wise (ROOT0).