◄ ENTELÉCHEIA · Book III · Circuits · the live tool

TOY MODELS OF SUPERPOSITION

live · 5 features → 2 dimensions · trained in your browser

The iconic experiment, reproduced live. A tiny autoencoder must store 5 features in only 2 dimensions. When features are dense, it can only keep the 2 most important — the rest collapse to zero. As you raise sparsity, it starts packing all five into the plane, overlapping them and tolerating interference: superposition. Watch the feature vectors organize from the axes into a pentagon.

The Feature Geometry

each arrow = one feature's 2D weight vector (color = importance); the plane is the 2 hidden dims
most importantmidleast important

WᵀW — the interference matrix

diagonal = how strongly each feature is stored; off-diagonal = overlap (interference) between features
+ (aligned)0− (opposed)
S = 0.00 · p = 1.00
slide to change sparsity — the model retrains live (Adam, importance-weighted MSE, sparse inputs)
feature probability
1.00
features represented
2 / 5
geometry
on the axes
loss
train step
0
What you're seeing. The model is out = ReLU((x·W)·Wᵀ + b) — it compresses x (5 sparse features) through 2 hidden dims and tries to reconstruct it, weighting each feature by an importance that decays 1 → 0.59. With dense inputs (S=0) it behaves like PCA: keep the top 2, drop the rest (you'll see 2 arrows on near-orthogonal axes, 3 collapsed to the origin). As sparsity rises, collisions become rare, so storing an extra feature — even with interference — pays off: the arrows fan out into a digon → triangle → pentagon. That packing of more features than dimensions is superposition — and it's why the inside of a network looks like a smear: more meanings than neurons. This is a from-scratch JS reimplementation of the experiment in Anthropic's Toy Models of Superposition notebook (Elhage, … Olah, 2022, MIT) — the math is re-derived and trained live; the result is theirs.