◄ WORLD V · SONNY 5DART 610 · a helldive into the mind

THE EMBEDDING meaning as a direction in space

To do math on words, give each one a vector — a point in a few hundred dimensions, learned so that similar meanings sit close together. The startling result: meaning becomes arithmetic. The direction from man to woman is the same as from king to queen, so king − man + woman ≈ queen. Embeddings are the input layer of every language model — the bridge from symbols to geometry.

THE TECHNIQUE king − man + woman ≈ queen

The demo runs the analogy on tiny hand-made vectors — king − man + woman lands exactly on queen: live demo







  
HISTORY & CREDIT word2vec · Mikolov 2013

  
“Word vectors store definitions.” — they store distributional meaning — a word is the company it keeps (Firth) — not dictionary definitions. cited

  
the space · each word → a vector, trained so context predicts the word.
the geometry · relations become directions: king − man + woman ≈ queen.
2013 · Mikolov et al., word2vec — the input layer of language models.

  
Symbols, turned into geometry. representation






  
RECOMMEND FOR I-13 the analogy, on the compiler

  
On i-13, with king=(3,1), man=(2,0), woman=(0,2), king−man+woman = (1,3) = queen exactly:

  
$ i13 run nn_embedding.i13
RUN OK · 36 step(s)
king - man + woman = (1, 3)
queen = (1, 3)
analogy_holds = 1

  
Recommend as a NULL — vector arithmetic (B39/B44). The analogy is pinned vector algebra over learned coordinates; the coordinates are a representation (B44). NULL — meaning as a direction in space.