An imperceptible nudge that flips the answer. For a linear classifier f(x)=sign(w·x+b) the story is exact: the signed margin is (w·x+b)/‖w‖, and the smallest push that flips the label is that distance travelled along −sign(margin)·w/‖w‖ — straight at the boundary, never sideways. Down the center the data flows: a point goes in, the engine finds the minimal flip, the perturbed point comes out. The blue team builds and defends it; the red team tries to break it.
source Szegedy, Zaremba, Sutskever, Bruna, Erhan, Goodfellow, Fergus, Intriguing Properties of Neural Networks (2013) — arxiv.org/abs/1312.6199. Rendered, not quoted.
The classifier is constructed, not trained, so every adversarial quantity has an answer you can check by hand: weights w = (3, 4), bias b = 0, hence ‖w‖ = 5. The decision boundary is the line where w·x+b = 0; the vector w points straight across it.
Two facts do all the work. (1) the minimal L2 perturbation has size equal to the geometric margin |w·x+b|/‖w‖. (2) its direction is ±w/‖w‖ — parallel to w. Move any other way and you waste distance.
| quantity | value (live) |
|---|
A decision boundary is nearer than the picture suggests. Under the-linear-representation, a network reads features as directions in a straight-line geometry — and that very linearity is the crack: along w the output climbs at rate ‖w‖ per unit step, so a push far too small for the eye is large for the machine.
The fragility is not a bug bolted on; it is the same geometry that lets the model generalize, read backwards. Each sphere is the next one's premise: the linear representation makes the adversarial example inevitable.
The blue team's live check: over a seeded batch of points, re-derive the minimal flip from the closed form and confirm a push of exactly the margin flips the label while a hair less does not, that the direction is collinear with w, and that ‖δ‖ equals the geometric margin. If red tampers, this badge is where it shows.
The input is a single point x = (x₁, x₂) in feature space, together with the fixed classifier (w, b). The weight vector w is the normal to the boundary: it names the one direction in which the answer changes fastest.
Everything the panel computes is a function of where x sits relative to that line. On one side w·x+b > 0 (class +1); on the other, < 0 (class −1). The distance to the line, in the units the model cares about, is the geometric margin — and it is exactly how far an attacker must travel.
Move a slider — the verdict and the minimal attack are computed from the closed form on the spot, never looked up.
What the machine produces, proven: the minimal L2 perturbation size is |w·x+b|/‖w‖, its direction is ∓w/‖w‖, and the optimal L∞ perturbation is ε·sign(w) (which flips once ε reaches |w·x+b|/‖w‖₁). Every number in the readout is one of these closed forms evaluated at the current x — the witness (left) re-derives them live; the red team (right) tries to make them lie.
Two more cracks. The "imperceptible" claim is measured in an Lp ball, which is a poor stand-in for human vision — some Lp-small changes are visible, some large ones are not. And Goodfellow's linearity explanation is a hypothesis, contested by boundary-tilting and non-robust-feature accounts. Adversarial examples are real; the single tidy cause is not settled.
"Adversarial examples are rare corner cases." Cut. They are dense — for a typical point almost every nearby direction toward the boundary works, and the boundary is close in the w-direction for most inputs.
"Models are fooled because they are too non-linear / overfit." Cut. Goodfellow et al. argued the opposite: excessive linearity in high dimensions is enough — the panel is the minimal proof.
"Any small perturbation can flip the label." Kept, corrected. Only the direction along w counts; a step orthogonal to w changes the score by exactly zero. Direction, not just size — which is what window 6 breaks.
The red team's move: swap the attack from the w-direction to a direction orthogonal to w and push hard. The score does not move, the label does not flip — so the blue team's witness (window 7), which insists the minimal push must flip, recomputes and turns red.
Redirect the perturbation orthogonal to w and the flip fails — proving it is the direction, not the size, that matters. Nothing is faked; the attack is real and it is caught.