Two arms · two tasks · real accuracy
Identical model capacity (one attention block + MLP + weighted pool + linear head, Adam). The plain arm attends over all N tokens with positions; the silo arm runs the same model over K k-means intents (size-weighted, so it keeps token frequency but loses order). Bars are test accuracy on 400 held-out examples; the red line is chance.
What's real — and what it isn't
What's real
The training is genuine gradient descent: selftest.py gradient-checks the hand-written backprop (analytic vs numerical, < 1e-5) — that's the anchor. Both arms are trained identically and evaluated on held-out data; the numbers above are what came out, reproducibly (seed 0). The silo's win on the set task and its collapse on the order task are both reported, unedited.
What it isn't
It is not a real-world benchmark and not a claim the silo is generally better. The tasks are synthetic probes chosen to isolate one variable (does order matter?); the plurality win reflects a cluster-shaped task. A real dataset, a tuned model, and error bars would be the next step — this is the honest, controlled version of "train it, test it."
model.py (forward + gradient-checked backward) · tasks.py · train.py (Adam → results.json) · selftest.py (grad check + the honest finding). The trilogy: build (v1) · measure (v2) · train & test (v3).