A gradient for functions with corners. At a smooth point a convex function has one tangent line; at a kink it has a whole fan of supporting lines, and every slope in that fan is a subgradient. For |x| at 0 the fan is exactly the interval [−1, 1]. Down the center a point and a candidate slope go in, the engine tests the supporting-line inequality, and the verdict — subgradient or not — comes out. The blue team builds and defends it; the red team tries to break it.
source R. T. Rockafellar, Convex Analysis, Princeton University Press (1970) — Part V, Directional Derivatives and Subgradients. doi.org/10.1515/9781400873173. Rendered, not quoted. AMBER classic print source, secondary link.
A vector g is a subgradient of a convex f at x₀ when the line through (x₀, f(x₀)) with slope g never rises above the graph:
f(y) ≥ f(x₀) + g·(y − x₀) for all y
The set of all such g is the subdifferential ∂f(x₀). At a smooth point it is a single number — the derivative. At a kink it is a closed interval of one-sided derivatives.
For the point on the panel below, its subdifferential and whether your slider slope lies inside it:
| x₀ | ∂|x| at x₀ | your g | inside? |
|---|
This is the-gradient-descent made to work where the gradient does not exist. Replace ∇f by any subgradient and the same step, x − t·g, still moves toward the minimum of a non-smooth convex f.
That single move unlocks the kink in the-lasso's |β| penalty and the corner of the ReLU — convex optimization without smoothness. Each sphere is the next one's premise: the derivative becomes a set, and optimization keeps running.
The blue team's live check: re-recover ∂|x| at 0 by scanning slopes, and re-test the engine's own claimed subgradient against the supporting-line inequality. If red claims a slope outside [−1, 1], its line rises above |x| and this badge turns red.
Two numbers feed the engine: a point x₀ on the graph of f(x) = |x|, and a candidate slope g for a supporting line at that point. The question the machine answers: is that line a genuine subgradient — does it stay on or below |x| everywhere?
At a smooth point (x₀ ≠ 0) exactly one slope works: the derivative sign(x₀). At the kink (x₀ = 0) a whole interval works. Feed the values into the panel below.
Amber = |x|. Dashed = your supporting line: green if it stays on/below |x|, red if it rises above.
Diminishing step tₖ = 0.5/√(k+1). The method has no true “descent” guarantee — only the best-so-far value provably converges, bounded by a closed-form rate.
Move either slider — the verdict is computed live from the supporting-line inequality on a grid of test points, never looked up.
What the machine proves, checked live at boot: ∂|x| at 0 is exactly the interval [−1, 1] — every slope inside is a subgradient, none outside is; at any x₀ ≠ 0 the subgradient is unique and equals sign(x₀); each subgradient's line is a supporting hyperplane lying on or below |x|; and the subgradient method with a diminishing step reaches the known minimizer x* = 0, honouring its closed-form convergence bound.
The blue team's witness (left) re-derives these live; the red team (right) tries to make them wrong.
In practice the structure is exploited instead: proximal-gradient / ISTA for the-lasso, or smoothing (Moreau — Nesterov) to recover a true gradient. The subgradient tells you a supporting line exists; it does not by itself tell you the fast way down.
“The subgradient is just the gradient, generalized to one vector.” Cut. At a kink it is a whole set — the subdifferential — a closed convex interval, not a single slope.
“Subgradient descent is a descent method.” Cut. The value can increase; only the distance to x* (proper step) and the best-so-far value provably fall. It is the subgradient method, not descent.
“Any slope near the corner counts at the kink.” Kept, corrected. Exactly the interval of one-sided derivatives [f′₋, f′₊] = [−1, 1] for |x| at 0 — and nothing outside it.
The red team's move: claim slope 1.5 — outside [−1, 1] — is a subgradient of |x| at 0. Its line 1.5·x rises above |x| for small positive x, so the inequality fails. The blue team's witness (window 7) is watching.
Assert a slope outside the fan and the supporting line breaks the surface. The witness recomputes the max violation, disagrees with the known [−1, 1], and turns red. Nothing is faked; the attack is real and it is caught.