content position v4 · DUAL
two silos in parallel · v4

DUAL SILO

v3 found one silo collapses on order. So v4 runs two silos in parallel and merges them at the end — a content silo (orderless) and a position silo (ordered bins) — same shared encoder, concatenated at the head. Trained from scratch, gradient-checked. The dual partially recovers the order the single silo threw away — measured, and reported short of full.

The held-out results

Three arms · two tasks · does a second view recover order?

The plain arm attends over all N tokens; the content silo over K k-means intents (orderless); the dual runs the content silo and a position silo (K ordered bins) through the same encoder and merges them at the head. Bars are test accuracy on 400 held-out examples; the red line is chance.

◎ results.json — trained from scratch; both the single and the DUAL backprop are gradient-checked in selftest.py
The dual's attention is 2·K² pairs — still half the plain model's N². The order-task recovery is partial: chunking into K bins is coarser than exact positions, so the dual lands between the single silo and plain. The plurality win is k-means' home turf. Synthetic probes, tiny models, one seed — a controlled ablation, reproducible with python train.py. Not a leaderboard.
The colophon

What's real — and what it isn't

What's real

The dual — two parallel silos through a shared encoder, merged at the head — is genuinely trained: selftest.py gradient-checks both the single and the dual backprop (< 1e-5). The recovery on the order task (single silo 42.5% → dual 61.8%) is real and reproducible (seed 0), driven by the position branch keeping order.

What it isn't

The recovery is partial, not full — 61.8% is well short of the plain model's 85%, shown side by side, not hidden. It is not a claim that dual silos beat transformers; a coarse K-bin position silo can't match true positions. Synthetic probes, tiny models — a clean ablation, not a benchmark.

SOURCE · github.com/DavidWise01/transformer-silo-v4 · model.py (shared encoder + single & dual heads, gradient-checked) · tasks.py (content + position silos) · train.py → results.json · selftest.py. The tetralogy: build (v1) · measure (v2) · train one silo (v3) · two in parallel (v4).
⧉ the code (GitHub) ⌗ v1 ↗ v2 ▲ v3 ◆ v5 · the router ⌂ ud0