A concept is not a neuron — it is a direction. And if concepts are directions, you can do arithmetic on meaning: take king, subtract the gender that separates man from woman, and you land exactly on queen. Here the embedding is built by hand so every readout has a closed form: two orthogonal axes (GENDER, ROYALTY), four words, and the analogy that closes to zero. Data flows down the center: words go in, the direction is recovered, the analogy comes out. Blue builds it; red entangles the axes to break it.
source Park, Choe & Veitch, The Linear Representation Hypothesis and the Geometry of LLMs (2023) — arxiv.org/abs/2311.03658; on Mikolov et al., Efficient Estimation of Word Representations (2013) — arxiv.org/abs/1301.3781. Rendered, not quoted.
Nothing is trained. Each word is a vector in R4: a shared content offset, plus a signed step along the GENDER axis and a step along the ROYALTY axis. A concept direction is recovered as the difference of class means — and it comes back parallel to the axis it was built from.
| word | vector | gender proj | royalty proj |
|---|
recovered GENDER dir · ROYALTY dir
One claim seeds a whole wing of interpretability: meaning lives in directions of activation space, not in single units. Grant that, and two neighbours follow directly.
the-activation-steering adds a concept direction to a forward pass to move behaviour; the-probing-classifier reads a concept off by projecting onto it. Steering and probing are the same vector, used two ways — both stand on this sphere. Each sphere is the next one’s premise.
The blue team’s live check: recompute the analogy residual and the axis orthogonality from the current model. Untampered they read zero. If red entangles the axes, this badge is where it shows.
The vocabulary is four words — man, woman, king, queen — and two concepts — GENDER and ROYALTY. Each word is a point; each concept is a labelled split of those points into a positive and a negative class:
| concept | + class | − class |
|---|---|---|
| GENDER | man, king | woman, queen |
| ROYALTY | king, queen | man, woman |
The hypothesis says a concept is a single direction. Feed the split to the panel: it recovers that direction as mean(+) − mean(−) and tests whether it truly separates and whether the axes compose by arithmetic.
| word | projection | class |
|---|
Every number is computed from the four word vectors on the spot — the difference of means, the projection, the analogy residual. Nothing is looked up.
What the machine produces, proven on the constructed data to 1e-9: the difference-of-means direction exactly separates each concept’s classes; king − man + woman = queen with zero residual (nearest neighbour queen); the concept direction is unique up to scale (any two class-mean differences are parallel); and projecting a concept out zeroes that coordinate and collapses its classes.
The blue witness (left) confirms these live; the red team (right) tries to make them false.
The famous king − man + woman result is weaker than folklore: the standard benchmark excludes the input words from the answer set, and the arithmetic often just returns a near neighbour rather than a truly derived point (Nissim et al., 1905.09866). Exactness holds here because the axes were built orthogonal.
“A neuron is a concept.” Cut. Concepts are directions in activation space, generally not axis-aligned to neurons — superposition packs many into few units (Elhage et al., 2022).
“Every concept is one direction.” Corrected. Most usefully are; but some are provably multi-dimensional (Engels et al., 2024). The hypothesis is linear, not one-per-scalar.
“word2vec proves analogies are exact algebra.” Cut. The eval hides the inputs and picks the nearest word; exactness is a property of a constructed orthogonal space, shown live in the panel — not of trained embeddings.
The red team’s move: entangle the axes. Tilt the gender axis of the royal words by a known 30° so GENDER and ROYALTY are no longer orthogonal. The concept still “works” loosely — but the arithmetic no longer closes.
Entangle, and king − man + woman stops landing exactly on queen — the residual jumps off zero and the 1e-9 check fails. The witness (window 7) recomputes and turns red. Nothing is faked; the attack is real and it is caught.