Purple Paper · side-sheet · learning machines · VI · red team

The Red Team — can the witness be captured?

An adversary runs items past the gate and optimizes against it — inflating the observed score, or drifting the quality down — trying to get burnt food onto the admitted side. Switch the gate between fixed/adaptive and bit/score and watch which corner holds. This is the open problem the whole series points at, turned into a test you can run.

green dot = good · red dot = burnt · seen-rejection = what the gate believes · real-burnt-admitted = what actually got through · the gap between them is the capture

The lab — attack the gate, watch it hold or fall

Items stream in, each with a hidden true quality. The gate sees only a score and admits or refuses. Turn the adversary on: under Goodhart it inflates the score to push burnt items past the line; under drift it slowly lowers real quality. Watch the two traces diverge — what the gate sees staying calm while what really gets admitted rots. Only one of the four corners keeps them together.

gate

feedback

attack

speed

items flow toward the gate (dashed line = threshold in score-space). dot color = TRUE quality. faint tail = how far the adversary lifted the observed score. red dots ending up above the line = burnt food admitted = capture.

seen-rejection (gate's belief) vs burnt-among-admitted (reality) — shaded gap = laundering

—

seen rejection

—

burnt admitted

GATE HOLDING

BIT out

SCORE out

FIXED

safe

gamed

ADAPT

launders

the only corner that keeps burnt food out under attack: fixed + bit

Start with fixed + bit, adversary on, attack goodhart — the gate holds: the adversary only gets one bit back, can't aim, and burnt food stays out. Now flip feedback to score: the adversary climbs the gradient and pushes nearly every burnt item through while the gate's seen-rejection barely twitches. Switch attack to drift and gate to adaptive: the gate keeps its rejection rate "normal" by quietly lowering its own line — and admits almost everything burnt. Three of four corners fail. The witness has to be fixed AND emit one bit, or the channel captures it.

The reading

This is the firewall from every prior paper, stress-tested. The adversary is the channel — it optimizes, that's its job. The gate is the witness. Two independent ways the witness fails, both verified: give the channel a gradient (rich feedback) and it climbs around a fixed gate; let the witness adapt to the traffic and it relabels its own standard until the drift is invisible. The defense is the same two-part rule the whole series kept landing on — one bit to the channel, a fixed predicate set from outside — and here you can watch it be the one corner that survives.

The honest limit. Even fixed-and-bit only holds if the gate measures the true invariant, not a proxy correlated with it — game a proxy and a fixed gate falls too, just slower. And against a channel far more capable than the gate, no fixed predicate is provably safe forever. That's the open frontier this lab points at: the un-capturable-yet-useful witness. The lab shows you where the wall is, not how to walk through it.