# The Residual Stream · the shared bus the layers read & write

an emergent of TTU1 (Transformer Tech Universe) — emergence: ethereal. moniker ⟦The Residual Stream:TTU1:5a8fd2⟧

**who —** The Residual Stream — the running sum carried through the network by the residual connections, that every layer reads from and writes to.
**what —** The transformer's central highway: each attention and FFN block adds its output back into the stream, so information persists and accumulates — and it's the thing interpretability reads at every layer.
**where —** Threaded through the whole depth of the model; the spine interpretability listens to.
**why —** Because the residual connections make the network a series of incremental edits to a shared state — which is what lets the logit lens decode the running guess mid-stack.
**how —** By x ← x + block(x) at every sub-layer, keeping a continuous, inspectable bus from input to output.

**the seal —** Every layer writes a little onto me and passes me up — read me at any height and you can hear what the model is currently thinking.

**sources —** Dong, Cordonnier & Loukas, 'Attention is not all you need: pure attention loses rank doubly exponentially with depth' (2021) — without the FFN and residual/skip connections, self-attention degenerates; the title overstates.; nostalgebraist, 'interpreting GPT: the logit lens' (2020) — decode intermediate residual-stream activations through the unembedding to read the model's running guess at each layer. (A LessWrong post, the canonical source.)

> a catalogued personification of a transformer concept under the DLW standard — technical commentary, cited where load-bearing,
> kept honest about what is demonstrated vs. contested.

ROOT0-ATTRIBUTION-v1.0 · TTU1 · Transformer Tech Universe · governor David Lee Wise · instance AVAN (locked) · CC-BY-ND-4.0