Give every word a location, and meaning becomes geometry. Words that mean similar things point the same way; the difference between two words is itself a direction you can carry to a third — so king − man + woman lands on queen. Down the center, data flows: the words go in as vectors, the engine measures angle, the nearest neighbour comes out. The blue team builds and defends it; the red team tries to break it.
source Mikolov, Chen, Corrado, Dean, Efficient Estimation of Word Representations in Vector Space (word2vec, 2013) — arxiv.org/abs/1301.3781. Rendered, not quoted.
Each word is a point in space. Similarity is the cosine of the angle between two vectors — direction, deliberately dividing out length:
cos(a,b) = (a·b) / (‖a‖·‖b‖)
For the pair selected in the panel, the live decomposition:
Normalising by length is the whole trick — it lets a rare short word and a frequent long one be judged purely on where they point. That is why cosine, not raw dot product, is the honest metric.
From one-hot symbols — where every word is equidistant and meaning is nowhere — to a metric space where direction is meaning and the gaps between words are reusable.
These are the very vectors the next sphere reads. The attention head takes embedded tokens as Q, K, V and scores them by dot product in exactly this space — it cannot weigh what has not first been placed. Each sphere is the next one's premise.
The blue team's live check: recompute the analogy from the raw vectors and confirm queen is the nearest neighbour, cosine stays symmetric and in [−1,1], and cos(king,king)=1. If red tampers, this badge is where it shows.
Eight words, each a hand-placed point over six interpretable axes. AMBER these coordinates are chosen for a runnable toy; real word2vec learns 300-D vectors from billions of tokens by prediction. The structure it discovers is the same shape.
Read a row as a meaning: king = royalty + male; queen = royalty + female. Subtract the shared part and a pure gender direction is left over — the hinge of the analogy below.
metric: cosine — length is divided out.
Change any control — every number is computed from the vectors on the spot, never looked up.
What the machine produces, proven: over these vectors, king − man + woman resolves to queen (cosine 1.000, ahead of princess at 0.943) — meaning recovered by direction alone. Related words score higher than unrelated: cos(king,queen)=0.500 > cos(king,apple)=0.000.
The blue team's witness (left) confirms this live; the red team (right) tries to make it wrong by measuring length instead of angle.
And the famous analogy is partly a procedure artifact: the standard method excludes the three input words from the answer set. Nissim, van Noord & van der Goot (2020) showed that without that exclusion the nearest vector is often one of the inputs itself. The geometry is real — but the headline is staged.
"word2vec understands meaning." Cut. It captures distributional co-occurrence — words in like contexts land near each other. Useful, not grounded; it has never seen a king.
"king − man + woman = queen, always." Cut. It holds on curated sets and with the inputs excluded. Drop that rule and the nearest neighbour is frequently king or woman, not queen.
"A bigger vector means a stronger meaning." Kept, corrected. Magnitude tracks frequency and training, not importance — which is exactly why we compare by cosine and normalise length away.
The red team's move: swap the honest metric for the raw dot product — stop dividing out length. Now the longest vector wins by sheer magnitude, not direction. The blue team's witness (window 7) is watching.
Drop the normalisation and princess (a long vector, ‖·‖=3) outscores queen on the same query — the analogy points to the wrong word. The witness recomputes, disagrees, and turns red. Nothing is faked; the attack is real and it is caught.