\o/ SEE INSIDE
a pocket universe the size of a forward pass

THE FORWARD PASS

A tiny traveler, \o/, rides the residual stream from embedding to logits — and at every station you see inside on the way through: the vector it carries, the attention it pays, the prediction it falls out as. A real toy transformer, computed live. The illustrative companion to crosstalk: the forward pass is where a marker can leak — so here it is, opened up.

Walk it through

Ride the residual stream

Step \o/ from station to station. It rides the last token of the sequence; each block reads the whole stream (causally) and adds to it. Watch the vector it carries change, and see what it looked at.

◎ a real forward pass · 4-dim residual · 1 head · 3 pre-norm blocks · deterministic, computed live
\o/

the residual stream

what \o/ looked at

Every number is really computed: RMSNorm → causal-softmax attention → MLP, each added back to the residual, then an unembed to logits over a tiny vocab. Fixed deterministic weights — the same run every time. This is the shape of a forward pass, honestly rendered; it is not a real language model. (And if you watch a moment: a second little traveler, ()\, drifts along behind the machine — head over the stations, tail below — while \o/ rides through it.)
The colophon

Real, and honestly small

What's real

A genuine forward pass in miniature: a 4-dimensional residual stream, one causal attention head whose weights are a true probability distribution, three pre-norm blocks, a real MLP, a real unembed. python selftest.py proves — no network — the attention is a distribution, the mask is causal, the pass is deterministic, and the stream really moves.

What it isn't

It is not a language model and makes no claim about any real model's internals. It shows the shape of the computation — what "through the weights" means — so the leak that crosstalk watches for has somewhere to live in your head. It detects nothing.

SOURCE · the runnable kit — github.com/DavidWise01/the-forward-pass · forward.py (the real toy pass) · selftest.py (distribution · causal · deterministic, no network). The page runs the same computation client-side.
⧉ the code (GitHub) ⇄ crosstalk · the inference membrane ⌗ transformer-silo · floor 2 is this engine ⊞ the membrane map ⌂ ud0 · the atlas