THE WEIGHT OF A WORD — WHAT THE SILO ACTUALLY SORTS
a build-on of THE WORD SILO · the same 33 real GPT-2 embedding norms, asked one honest question: what does that "weight" track? · tier: lit (correlations computed in-page) · 2026-07-10
The silo confines each word to a layer by its embedding norm and calls it weight. Fair — but a norm is just a length in a 768-d space; the word "weight" is a reading. So on the same 33 words, measured, not asserted: the norm sorts register (function words light, rare content words heavy, r≈0.86) with length riding along (r≈0.76, because function words are short) — the two are entangled. It is not semantic mass. The verified anchor: the eight lightest words are exactly the eight function words.
NORM vs LENGTH · colored by register
■ function words■ content words · x = weight (norm, normalized) · y = word length. Both rise together — but the register split is the cleaner cut than length alone.
THE STACK, RE-READ · light → heavy
the 8 lightest (bottom) are the 8 function words; the heaviest are rare content words (rune / vortex / phantom). "weight" is register + rarity, riding on length.