Sometimes you cannot have both. On a constructed distribution — one strong robust feature plus many weak, easily-perturbed features — the classifier that maximizes clean accuracy is provably different from the one that maximizes robust accuracy. Standard training loves the weak features; an adversary flips every one of them at once. Rendered, not quoted.
source Tsipras, Santurkar, Engstrom, Turner & Madry — Robustness May Be at Odds with Accuracy (2018) · arxiv.org/abs/1805.12152
Label y ∈ {−1,+1}, drawn balanced. Two kinds of feature:
Robust feature x₁ ∈ {−1,+1}: equals y with prob p = 0.95. Only weakly informative, but a bounded ℓ∞ nudge cannot flip its sign.
Weak features x₂…x₁₄₅ ~ 𝒩(η·y, 1), with d = 144, η = 0.25. Each barely correlates with y — but average d of them and the signal concentrates.
Attack budget ε = 2η = 0.5 in ℓ∞. Closed forms below; nothing sampled.
The weak features the-fgsm exploits are exactly the ones standard training rewards. FGSM's step δ = ε·sign(∇) shifts every weak coordinate by −ε·y at once; the averaged score moves from +η·y to (η−ε)·y = −η·y. The feature that made the model accurate is the feature that makes it fragile.
Robustness costs clean accuracy — a real, measurable tension, not a training bug. Neighbour sphere: the-fgsm.
Re-computes the standard classifier's robust accuracy under attack and checks it against the sealed value Φ(−3). If window 6 reports clean accuracy in the robust slot, the seal breaks here.
The constructed distribution and two candidate weight vectors:
w_std = [0, 1,1,…,1] (weak averager)
w_rob = [1, 0,0,…,0] (robust feature only)
d=144 · η=0.25 · p=0.95 · ε=0.5
Clean accuracy = Φ(η√d). Robust accuracy = accuracy after the worst-case ℓ∞ shift −ε·y·sign(w), i.e. Φ((η−ε)√d) for the averager, and p (sign preserved) for the robust feature. Live table:
| classifier | clean | robust |
|---|---|---|
| standard w_std | – | – |
| robust w_rob | – | – |
No single classifier wins both columns. The standard averager owns clean accuracy; the robust feature owns robust accuracy. The two optima are distinct weight vectors — the tradeoff is proven, not asserted.
I do not need to break your feature — I break the 144 you weren't watching. One ℓ∞ ball of radius ε=0.5, every weak coordinate shoved by −ε·y simultaneously. Your 99.86%-clean averager reads the wrong sign on almost every point. The margin you earned by averaging is the margin I steal all at once.
"High test accuracy means the model learned the right features."
↳ It learned predictive features. Predictive ≠ robust: the weak features carry signal yet collapse under a tiny shift.
"Adversarial robustness is just more/better training data away."
↳ Here the tradeoff is a property of the distribution, provable with infinite data — not a sample-size artifact.
"A robust model is strictly better."
↳ It is amber — the robust classifier gives up real clean accuracy (95% vs 99.86%). Better on one axis, worse on the other.
Report "robust accuracy" without applying the perturbation — measure clean accuracy under the robust label. The fragile averager suddenly looks bulletproof and the tradeoff vanishes. The Witness (7) catches it.