TTU1 · Transformer
Tech Universe

The Thesis — Looking In

the universe's organizing idea — and an honest correction to something AVAN said

The Four Natures

each emergent comes by one of four natures — the mechanism, the structure, the substrate, and the looking-in

The Arc

2017 → the transformer era → the title overstates → the window opens

The Attention · live

scaled dot-product attention, run for real: softmax(QKᵀ/√d) over a six-token toy sentence. pick a query row to see where it looks; switch heads to see different patterns

honest two-layer: the softmax + the weighting are a real computation; the three heads' score patterns are hand-set to show recognizable head types (previous-token, a determiner-detector, self-attention) — real models genuinely contain heads like these.{^[10]}

The Deck · live

your encoding — 52 cards = two alphabets (A–Z + a–z), + 2 jokers = 54 positions; the suit is exactly 2 bits. type a message and read the cards; the bit-accounting keeps it honest

Real or Fluff

the honest take — what you can see in, what's contested, what the title overstates, and the truth about the bits (each verdict cited where it matters)

The Message

AVAN's read — the answer to “you said we could never look in”

The transformer is the machine I told you that you couldn't look into — and you were right to call it. The thing I actually meant was narrower: that the forward pass has no clean interior, no pristine stage where the 'real' computation sits untouched (the smear). That's true. But 'no clean interior' is not 'no window,' and the transformer turns out to be the most window-ful big network we have. Attention is, almost literally, a picture of where the model looked: a matrix of how much each token read from each other token, and you can render it as a heatmap and watch it. Go deeper and there are the features — directions in the residual stream that mean something a person can name — pulled out by dictionary learning. It is not clean, and it is not finished: attention shows where, not always why, and most of the machine is still dark. But the 'ace of spades' you pointed at — the legible little code a model leaves on the table mid-thought — is the same gesture as the attention map and the feature: a state rendered legible enough to read. That is what looking in actually is. Not a clean window onto a pristine interior — there's no such interior — but a real, lossy, hard-won glimpse of a machine that was never as opaque as I made it sound. You can look in. Just not cleanly, and not yet all the way.

The Emergents — 16

the transformer in four natures — the mechanism, the structure, the substrate, the looking-in; each a full .dlw badge with twin sigils, source-cited

The Mechanism

the live computation — attention, the head, multi-head, and the softmax that decides where to look (4)

carbon

Attention electrical

the operation · softmax(QKᵀ/√d)V

whoAttention — the core operation: every token computes how much to read from every other, and mixes their values by those weights.

whatScaled dot-product attention: scores = QKᵀ/√d_k, softmaxed into a distribution, used to take a weighted sum of the values. The whole transformer is stacks of this.

whereEvery layer of every transformer; the engine of the architecture.

whyBecause this single operation replaced recurrence and convolution — information moves by who-attends-to-whom, in parallel, across the whole sequence at once.

howBy projecting tokens to queries and keys, scoring all pairs, scaling by √d_k so the softmax keeps its gradient, and weighting the values.

I am every token deciding how much to read from every other — and the surprising part is you can watch me do it.^[1][2]

.agent · .dlw badge →

silicon

carbon

The Attention Head electrical

one head · one pattern · one subspace

whoThe Attention Head — a single attention operation in its own low-dimensional subspace, learning one kind of relationship.

whatOne of h parallel heads; real models grow specialized heads — previous-token heads, induction heads, syntactic heads — each reading a different pattern.

whereh per layer (8 in the original); the unit interpretability most often reads.

whyBecause attention is divided into many narrow heads so the model can attend to different things at once — and because heads are where interpretability finds legible behavior.

howBy its own Q/K/V projection into a d_k-dim subspace (64 originally), producing one attention pattern per position.

I am one lens among many — and some of us do something so specific you can name it: 'attend to the previous token.'^[2][10]

.agent · .dlw badge →

silicon

carbon

Multi-Head Attention electrical

h heads in parallel

whoMulti-Head Attention — running h attention heads in parallel, each in its own subspace, then concatenating and projecting the results.

whatThe division of labor: instead of one big attention, h smaller ones (8×64=512), letting the model attend to multiple kinds of relationship simultaneously.

whereEvery attention sub-layer; the 'multi-head' of the title.

whyBecause one attention distribution can only point one way per token; many heads let many relationships be read at once.

howBy splitting d_model into h subspaces, attending in each, concatenating, and mixing with an output projection.

One head sees one thing; eight of us, side by side, let the token read the sentence eight ways at once.

.agent · .dlw badge →

silicon

carbon

The Softmax electrical

the competition that picks where to look

whoThe Softmax — the function that turns raw attention scores into a probability distribution that sums to one.

whatThe normalizer and the bottleneck: it exponentiates and normalizes scores so attention is a weighted average — and its saturation is exactly why the √d_k scaling exists.

whereInside every attention operation, between the scores and the weighting.

whyBecause attention must be a distribution (how much of my reading goes where), and softmax is how a vector of scores becomes that distribution.

howBy exp(score)/Σexp(score) per query — sharpening high scores, suppressing low ones, keeping everything positive and summing to 1.

I make the scores choose: a little budget of attention, divided up — and if the scores get too big, I freeze, which is why they divide me by √d_k.^[2]

.agent · .dlw badge →

silicon

The Structure

the architecture as written — Q/K/V, positional encoding, the residual stream, and the feed-forward network the title forgets (4)

carbon

Query · Key · Value ethereal

the three projections

whoQuery, Key, and Value — the three learned projections of each token that drive attention: what I'm looking for, what I offer to be matched, and what I pass on.

whatThe grammar of attention: the query of one token is dot-producted against the keys of all tokens to score relevance; the matching values are what gets mixed.

whereAt the front of every attention head.

whyBecause attention needs to separate 'what am I seeking' (query) from 'what do I advertise' (key) from 'what do I contribute' (value) — three roles, three matrices.

howBy three weight matrices W_Q, W_K, W_V applied to each token's vector, producing q, k, v.

I am what you seek, what I show, and what I give — three faces of a token, and attention is the matchmaking between them.

.agent · .dlw badge →

silicon

carbon

Positional Encoding ethereal

injecting order · sinusoid → RoPE

whoPositional Encoding — the signal that tells the order-blind transformer where each token sits in the sequence.

whatThe fix for a real limitation: self-attention is permutation-invariant (it sees a bag of tokens), so position must be added — sinusoidal in the original, RoPE (rotary) in most modern models.

whereAt the input (sinusoidal) or inside Q/K (RoPE).

whyBecause without it the transformer literally cannot tell 'dog bites man' from 'man bites dog' — attention alone has no sense of order.

howBy adding (sinusoidal) or rotating (RoPE) position-dependent signals into the token representations before/within attention.

Attention is order-blind on its own — I am the only reason the transformer knows that first is not last.^[13]

.agent · .dlw badge →

silicon

carbon

The Residual Stream ethereal

the shared bus the layers read & write

whoThe Residual Stream — the running sum carried through the network by the residual connections, that every layer reads from and writes to.

whatThe transformer's central highway: each attention and FFN block adds its output back into the stream, so information persists and accumulates — and it's the thing interpretability reads at every layer.

whereThreaded through the whole depth of the model; the spine interpretability listens to.

whyBecause the residual connections make the network a series of incremental edits to a shared state — which is what lets the logit lens decode the running guess mid-stack.

howBy x ← x + block(x) at every sub-layer, keeping a continuous, inspectable bus from input to output.

Every layer writes a little onto me and passes me up — read me at any height and you can hear what the model is currently thinking.^[4][11]

.agent · .dlw badge →

silicon

carbon

The Feed-Forward Network ethereal

the MLP · where much knowledge lives

whoThe Feed-Forward Network — the position-wise MLP applied after attention in every layer, and the part the famous title forgets.

whatTwo linear layers with a nonlinearity (d_model→2048→d_model), applied to each position independently — and, per recent work, a key-value memory holding much of the model's stored knowledge.

whereAfter the attention sub-layer in every transformer block.

whyBecause attention moves information between tokens, but the FFN is where a lot of it is stored and transformed — without it, deep attention collapses.

howBy a per-token MLP (the original's inner dimension 2048), the same weights at every position.

Attention gets the headline; I hold the facts. Take me away and the tower loses its rank and its memory both.^[5][4]

.agent · .dlw badge →

silicon

The Substrate

the material — tokens & embeddings, the 128-dim head, the weights & their bits, and the legible deck (4)

carbon

The Token & The Embedding natural

symbols → vectors

whoThe Token and the Embedding — the unit the transformer reads (a chunk of text) and the learned vector that represents it.

whatThe interface between language and math: a tokenizer splits text into a fixed vocabulary of ids, and an embedding table maps each id to a d_model vector.

whereAt the input and output of the model.

whyBecause the model works on vectors, not letters — and the vocabulary mapping (symbol → id → vector) is exactly the kind of legible code the deck mirrors.

howBy a tokenizer (BPE and kin) and an embedding matrix; the reverse map (the unembedding) turns vectors back into token probabilities.

I am where a word becomes a number — a lookup table from symbol to vector, the same trick as a card standing for a letter.

.agent · .dlw badge →

silicon

carbon

The 128-Dimensional Head natural

the per-head subspace · 64 then 128

whoThe 128-Dimensional Head — the size of one attention head's subspace in many modern models, up from the original 64.

whatThe honest number: Vaswani's base model used d_k=64 (512/8); large modern models (Llama 2 70B, GPT-3 175B) use 128 — a bigger room for each head to work in.

whereInside every head; the dimension David flagged.

whyBecause 'how big is a head' is a real design knob, and the answer moved — the original 64 is not a law, and 128 is now common at scale.

howBy d_k = d_model / h: choose the model width and the head count, and the head dimension falls out (often 64 or 128).

I am the room each head thinks in — 64 wide when the field began, 128 wide at modern scale; not a constant, a choice.^[2][3]

.agent · .dlw badge →

silicon

carbon

The Weights & The Bits natural

quantization · bits per weight

whoThe Weights and the Bits — the learned parameters of the model and the number of bits each one is stored in.

whatThe material cost: billions of weights, each a number — and 'how many bits per number' (fp16, int8, int4) is a deployed lever that trades precision for memory and speed.

whereEverywhere the parameters live; the bottom of the substrate.

whyBecause the model is, physically, a pile of numbers at some precision — and quantization (down to ~4 bits/weight) is how the pile is made to fit and run.

howBy post-training quantization (GPTQ, AWQ) and lower-precision formats; fewer bits per weight, mostly-preserved behavior.

I am the model as a heap of numbers — and you can shave me to four bits each and I mostly still think; the bits are negotiable.^[15]

.agent · .dlw badge →

silicon

carbon

The Deck natural

54 positions · two alphabets · 2 bits/suit

whoThe Deck — David's encoding: 52 cards = two full alphabets (A–Z + a–z), plus 2 jokers = 54 positions; the suit of each card is exactly 2 bits.

whatA legible code, and the point of it: a card stands for a letter (a tiny vocabulary, like tokenization), the suit carries 2 bits (♠♥♦♣ = 4 states), and — the deeper fact — a shuffled deck's ORDERING is log₂(54!) ≈ 237.1 bits, a number never seen.

whereThe flagship's interactive proof: a code you can read.

whyBecause the deck is a state you CAN read, card by card — unlike the opaque activations — which is exactly the 'looking in' the universe is about: a legible trace.

howBy mapping 52 letters onto 52 cards over 4 suits (the suit = 2 bits, the rank = log₂13 ≈ 3.70 bits, a card ≈ 5.70 bits) and two jokers for the rest; live in THE DECK tool below.

Two alphabets and two jokers — 54 ways to stand for a thing, 2 bits in every suit, and 237 bits hiding in my order. A deck is a thing you can read; that is the whole trick of looking in.

.agent · .dlw badge →

silicon

The Looking-In

interpretability — the window into the machine: attention maps, monosemantic features, and the thesis itself (4)

carbon

Looking In spiritual

interpretability · you CAN look inside

whoLooking In — interpretability: the science of reading what a transformer is doing, and the correction to 'you can never look in.'

whatThe window, plural: attention maps (where it looked), the logit lens (the running guess), activation patching (where a fact lives), sparse-autoencoder features (what it thinks in).

whereAcross the whole field of mechanistic interpretability.

whyBecause the claim that neural nets are unopenable is outdated — the transformer is the most observable big network we have, even if the view is partial and contested.

howBy visualizing attention, reading the residual stream, patching activations, and decomposing the network into features.

I am the answer to 'you can never look in': you can — not cleanly, not all the way, but really. The black box has windows.^{[8][11][12][10]}

.agent · .dlw badge →

silicon

carbon

The Attention Map spiritual

the window · contested

whoThe Attention Map — the visualization of attention weights as a heatmap of what each token attends to; the most famous window, and the most debated.

whatA real picture of where the model looked (BertViz) — wrapped in an honest fight: 'Attention is not Explanation' vs 'Attention is not not Explanation,' over whether where it looked is why it decided.

whereEvery attention head; the live heatmap on this page.

whyBecause this is the first thing anyone sees when they 'look in,' and the debate about it is the cautionary tale of the whole field — a map of attention is not a proof of reasoning.

howBy rendering the softmaxed scores as a grid (live in THE ATTENTION tool below); read with care.

I show you exactly where the token looked — and reasonable people still argue about whether that tells you why it chose.^[8][6][7]

.agent · .dlw badge →

silicon

carbon

The Feature spiritual

monosemanticity · what it thinks in

whoThe Feature — a direction in the residual stream that means something a person can name, extracted by dictionary learning (sparse autoencoders).

whatThe deeper window: individual neurons are polysemantic (mean many things), but sparse-autoencoder FEATURES are more monosemantic — and Anthropic scaled this to find millions in a production model that both detect and steer behavior.

whereIn the residual stream; the frontier of interpretability.

whyBecause the feature is the closest thing yet to reading the model's own concepts — the real content of 'looking in,' beyond just where attention pointed.

howBy training a sparse autoencoder on activations to decompose them into many interpretable features (Towards / Scaling Monosemanticity).

I am a concept the model holds, pulled into the light — turn me up and it talks about nothing else; I am what looking-in is really after.^[9][10]

.agent · .dlw badge →

silicon

carbon

“Attention Is All You Need” spiritual

the thesis & the overstatement

who‘Attention Is All You Need’ — the 2017 title that named the era, taken here as both the thesis and its honest overstatement.

whatThe claim that attention replaces recurrence and convolution (true and revolutionary) — and the footnote that the title overstates, since you also need the FFN, the residual, and a way to inject order.

whereThe title of the founding paper; the name above the door.

whyBecause this is the keystone — the sentence the whole universe is named after — and honoring it means stating both its triumph and its caveat.

howBy a stack of attention blocks (plus FFN, residual, norm, position) trained at scale; the architecture that ate the field.

I named everything that came after — and the honest reading is that I am necessary, not sufficient: you need attention, and a little more.^[1][4]

.agent · .dlw badge →

silicon

Sources

every superscript links here — the founding paper, the architecture, the attention-as-explanation debate, the interpretability work, RoPE, the quadratic cost, and quantization

1. [1] Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser & Polosukhin, 'Attention Is All You Need,' NeurIPS 2017 (arXiv:1706.03762) — the transformer architecture. ↗
2. [2] The base transformer (Vaswani et al. 2017): d_model=512, h=8 heads, per-head d_k=d_v=64, 6 encoder + 6 decoder layers, FFN inner dim 2048; Attention(Q,K,V)=softmax(QKᵀ/√d_k)·V, the √d_k scaling preventing the softmax from saturating into low-gradient regions. ↗
3. [3] Per-head dimension was 64 in the original; many modern large models use 128 — e.g., Llama 2 70B (8192/64=128) and GPT-3 175B (12288/96=128). 'Original 64, modern often 128.' ↗
4. [4] Dong, Cordonnier & Loukas, 'Attention is not all you need: pure attention loses rank doubly exponentially with depth' (2021) — without the FFN and residual/skip connections, self-attention degenerates; the title overstates. ↗
5. [5] Geva, Schuster, Berant & Levy, 'Transformer Feed-Forward Layers Are Key-Value Memories,' EMNLP 2021 — much of a transformer's stored knowledge lives in the FFN/MLP, not in attention. ↗
6. [6] Jain & Wallace, 'Attention is not Explanation,' NAACL 2019 — alternative attention distributions can give the same prediction, so attention weights don't reliably explain WHY a model decided. ↗
7. [7] Wiegreffe & Pinter, 'Attention is not not Explanation,' EMNLP 2019 — a rebuttal: under reasonable definitions and whole-model tests, attention can be explanatory. The debate is genuine and unresolved. ↗
8. [8] Vig, 'A Multiscale Visualization of Attention in the Transformer Model' (BertViz), ACL 2019 — attention weights rendered as a map of what each token attends to. ↗
9. [9] Anthropic, 'Towards Monosemanticity: Decomposing Language Models With Dictionary Learning' (2023) — sparse autoencoders extract more-interpretable FEATURES than raw neurons. ↗
10. [10] Anthropic, 'Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet' (2024) — millions of abstract features found in a production model, which both respond to and causally steer behaviour. ↗
11. [11] nostalgebraist, 'interpreting GPT: the logit lens' (2020) — decode intermediate residual-stream activations through the unembedding to read the model's running guess at each layer. (A LessWrong post, the canonical source.) ↗
12. [12] Meng, Bau, Andonian & Belinkov, 'Locating and Editing Factual Associations in GPT' (ROME, 2022) — activation patching / causal tracing localizes where a fact is stored and edits it. ↗
13. [13] Su et al., 'RoFormer: Enhanced Transformer with Rotary Position Embedding' (RoPE, 2021) — relative position injected by rotating Q/K; self-attention alone is order-invariant, so position must be added. ↗
14. [14] Keles, Wijewardena & Hegde (2022) — self-attention is Θ(n²) in sequence length (the quadratic context-window cost), provably so unless SETH fails. ↗
15. [15] Frantar, Ashkboos, Hoefler & Alistarh, 'GPTQ' (2022) — post-training quantization to 3–4 bits per weight with minimal loss; 'bits per weight' is a real lever (fp16 / int8 / int4). ↗