Transformer Inference · Instrument 00 · Primer

From text to the next word

A language model does one thing: predict the next token. But the path from your words to that prediction runs through a remarkable machine — text becomes numbers, numbers become vectors, vectors trade information through attention, refine across dozens of layers, and finally collapse into a single guess. This is that forward pass, end to end, plus the backward pass that taught the machine in the first place.

01Tokenization: text becomes pieces

A model can't read letters — it reads tokens, chunks of text drawn from a fixed vocabulary. Common words are single tokens; rarer ones split into subwords. Each token is just an integer ID, an index into the vocabulary.

A sentence split into tokens — note how Transformers breaks into pieces. Each gets an ID.

02Embeddings: pieces become meaning

Each token ID is looked up in a table, becoming a vector — a list of numbers, a point in a high-dimensional space. The model learns to place related meanings near each other, so much so that directions carry meaning: the step from man to woman is the same step as king to queen.

Words as points. king − man + woman lands right on queen — meaning encoded as geometry.

03Attention: pieces share context

A word's meaning depends on its neighbors — "bank" by a river isn't "bank" with money. Attention lets every token look at every other and pull in what's relevant, weighting each by how much it matters. Those weights always sum to one: a token spends its full attention, distributed.

The current token attends to all others; arc thickness is the weight. Each token gathers its context.

04Layers: meaning gets refined

The vectors flow upward through a stack of identical layers, each one running attention and a small neural network, each writing its result back into a shared residual stream. Information accumulates: early layers catch grammar, later ones catch sense. Modern models stack dozens to hundreds.

A vector rising through layers — each adds attention and MLP contributions to the residual stream.

05The output: a guess at the next token

At the top, the final vector is projected onto the whole vocabulary, scoring every possible next token — the logits. A softmax turns those scores into probabilities that sum to one, and the model samples one. That token is appended, and the whole process runs again for the next.

Logits → softmax → probabilities → pick. "The cat sat on the ___" resolves to its likeliest continuation.

06The reverse pass: how it learned

None of this works until the weights are right — and they start random. During training the model predicts, measures how wrong it is with a loss, then sends that error backward through every layer, nudging each weight in the direction that reduces it. Repeat across trillions of tokens and the machine above emerges.

Gradients flow backward; weights step downhill; the loss falls. This is gradient descent — learning, made mechanical.