Plain gradient descent zig-zags miserably down a steep-sided ravine, bouncing off the walls. Give it MOMENTUM — let each step keep a fraction of the last, like a heavy ball rolling — and the side-to-side bounces cancel while the downhill speed builds. It reaches the bottom in a fraction of the steps. It’s why real optimizers (Adam, SGD+momentum) all carry velocity. Slide the momentum.
Momentum accumulates a velocity that is an exponentially-decaying average of past gradients: v ← βv − η∇L, θ ← θ + v. In an ill-conditioned ravine (large curvature in one direction, small in another) plain GD oscillates across the steep walls; momentum cancels those oscillations while accelerating along the shallow valley floor — often an order-of-magnitude fewer steps. It is the core of SGD-with-momentum, Nesterov, and (with per-parameter scaling) Adam. A fail-loud self-check throws unless momentum reaches the minimum of a ravine in fewer steps than plain GD. ◆ real optimization, node-verified.
Too much momentum (β→1) overshoots and can become unstable; the optimal β depends on the curvature. The ravine speedup and the velocity update are exact. Shown on a fixed quadratic ravine.