Optimization on a leash. To minimize f while pinned to a constraint g = 0, you do not walk downhill — you walk along the curve until the descent you want points straight through the wall. At that spot the two gradients line up: ∇f = λ∇g. The scalar λ is the exchange rate — exactly how much the best value would improve if you loosened the leash by one unit. Data flows down the center: the objective and the constraint go in, the stationary condition is solved, the optimum and its multiplier come out. Blue builds and defends; red tries to break it.
source Lagrange, Méchanique analitique (Paris, 1788) — archive.org/details/meychaniqueanal00lagr. Method of multipliers, §IV. Secondary, modern: Boyd & Vandenberghe, Convex Optimization (2004) AMBER. Rendered, not quoted.
Fold the constraint into the objective. Form L(x,y,λ) = f − λ·g and demand it be stationary in all three variables:
∂L/∂x = 0 → fx = λ gx · ∂L/∂y = 0 → fy = λ gy · ∂L/∂λ = 0 → g = 0. The last equation is just the constraint, handed back. The first two say ∇f is parallel to ∇g.
Live at the current optimum — the two gradients and their single common ratio λ:
| ∇f | ∇g | ∇f / ∇g |
|---|
One idea — at the optimum the gradients align — turns a constrained problem into a stationary one you can solve with algebra. Widen the leash from an equality to an inequality and the same trick grows a sign and a slackness rule: ∇f = λ∇g, λ ≥ 0, λ·g = 0.
That is the Karush–Kuhn–Tucker conditions, and dualizing λ itself is Lagrangian duality. Each sphere is the next one's premise.
The blue team's live check: re-solve a battery of constrained problems and confirm, for each, that g = 0 holds at the optimum, the gradients are parallel, and λ equals the measured shadow price d f*/dc. If red drops the leash, this badge is where it shows.
A constructed convex problem — a round bowl and a straight wall:
f is strictly convex (a paraboloid); g is a line. There is exactly one constrained minimizer — the point of the line nearest the origin. Every optimum, multiplier and rate below is closed-form: x* = ac/(a²+b²), y* = bc/(a²+b²), λ = 2c/(a²+b²). Feed a, b, c into the panel.
Solve ∇f = λ∇g together with g = 0. The optimum is computed from the two gradients on the spot — never looked up.
What the machine produces, proven: the constrained minimizer ·, the optimal value ·, and the multiplier ·. The multiplier is not decoration — it is the shadow price: relax the constraint to c+Δ and the optimum improves by exactly λ·Δ. Measured d f*/dc = ·, matching λ.
Blue's witness (left) confirms these live; red (right) tries to make them wrong.
And it assumes a constraint qualification: ∇g ≠ 0 at the solution. Where the constraint has a cusp or ∇g vanishes, no λ exists and the method silently fails. Equality only in this basic form — inequalities need the sign and slackness of KKT.
"Lagrange multipliers find the minimum." Cut. They find stationary points. It takes second-order conditions — or, as here, convexity — to know a given one is a minimum and not a saddle.
"λ is a dummy variable with no meaning." Kept, corrected. λ is the shadow price d f*/dc — the marginal value of the constraint. The engine measures it numerically and it matches.
"The method works for any constraint." Cut. It needs a constraint qualification (∇g ≠ 0 at the solution). Fail that and no multiplier exists.
The red team's move: ignore the constraint and just minimize f — the unconstrained minimizer is the origin (0,0). It has the smaller f, but it is off the wall: g ≠ 0, and the gradients no longer align.
Drop to the unconstrained min and the on-constraint check fails — the witness (window 7) recomputes, finds g ≠ 0, and turns red. Nothing is faked; the attack is real and it is caught.