A layer crams more features than it has neurons — superposition — and each neuron lights for many unrelated things (polysemanticity). The sparse autoencoder answers with a wider, overcomplete dictionary and a sparsity penalty: pull the crammed features back apart into atoms that each mean one thing. Down the center: an activation goes in, convex sparse coding (ISTA) recovers the code, the monosemantic result comes out. The blue team builds it on a dictionary with known ground truth; the red team tries to break it.
source Bricken et al., Towards Monosemanticity: Decomposing Language Models With Dictionary Learning (2023), Anthropic / Cunningham, Ewart, Riggs, Huben, Sharkey, Sparse Autoencoders Find Highly Interpretable Features in Language Models (2023) — arxiv.org/abs/2309.08600. Rendered, not quoted.
The generative story is exact: an activation is x = D c*, where D is a dictionary of unit feature-directions and c* is sparse — only a few features fire. Recovery is the convex program
minc ½·‖x − Dc‖² + λ·‖c‖1
solved by ISTA: gradient step, then soft-threshold shrink(·, λ·t) — the exact proximal operator of the L1 term. It provably descends the objective (step ≤ 1/L). Here D is overcomplete (K=8 atoms in N=6 dims — superposition), built by hand so every readout is closed-form:
| atom | direction | max |cos| to another |
|---|
The neighbouring sphere THE SUPERPOSITION shows the crime: a network packs more features than neurons into nearly-orthogonal directions, so single neurons go polysemantic. This sphere is the inverse operation — dictionary learning that unpacks that overcomplete code.
One deletion carries between them: superposition hides features in overlap; the sparse autoencoder recovers them by betting the true code is sparse. It is the tool underneath monosemantic interpretability — each sphere is the next one's premise.
The blue team's live check: on the canonical test signal, re-solve and confirm the recovered code is sparse (L0 = 2) with exact reconstruction. If red deletes the L1 penalty, this badge is where the dense code shows.
A synthetic activation: a sparse combination of known ground-truth feature directions. Toggle which features fire below — the machine reads only the resulting vector x, not which atoms you chose. Recovering that hidden support is the whole task.
Two features on by default (f2, f6) — a genuinely sparse cause, superposed into the N=6 activation space.
Sparse coding: the L1 penalty forces most atoms to exactly zero. ISTA converges monotonically; the support is then debiased by least squares.
Every number is computed on the spot — ISTA on the known dictionary, then a least-squares debias on the recovered support. Nothing is looked up.
What the machine produces, proven: a sparse code whose support is exactly the features that fired, each active atom aligned |cos| = 1 with one ground-truth feature (monosemantic), and reconstruction error under 1e-3 after debias. The current signal's code is above; the invariants are the output.
The blue team's witness (left) confirms sparsity live; the red team (right) deletes the penalty and the code goes dense.
L1 also biases magnitudes toward zero (why this panel debiases). And the claim that a recovered atom is a real feature the model uses is AMBER — assumed, not proven. What is LIT here is only the convex recovery on a dictionary whose ground truth we constructed.
"An SAE gives you the model's true features." Cut. It learns a dictionary that reconstructs activations sparsely — a useful basis, not the network's privileged units. Feature splitting and absorption are documented.
"Monosemantic means one human concept." Kept, corrected. Monosemantic = fires on one coherent pattern by the autoencoder's lights. The human label is an interpretation, sometimes too broad or wrong.
"Sparse coding is convex, so it's all solved." Corrected. Solving the code for a fixed dictionary is convex (this ISTA). Learning the dictionary is not — that is the hard, non-convex part.
The red team's move: set the L1 penalty to zero. With no sparsity pressure, the overcomplete solver spreads the code across correlated atoms — the code goes dense, the atoms go polysemantic. The witness (window 7) is watching.
Delete the penalty and the min-norm solution smears feature f6 across atoms f0 and f1 — L0 jumps from 2 to 4, the sparsity check disagrees with ground truth, and the witness turns red. Nothing is faked; the attack is real and it is caught.