Every neural network learns by ONE idea: measure the slope of your error, and step downhill. Gradient descent nudges each parameter opposite its gradient, shrinking the loss a little each step, until it settles at the bottom. Do it a billion times over a trillion weights and you have a trained model. Slide the step size and watch the ball roll to the minimum.
Gradient descent minimizes a loss L(θ) by repeatedly stepping against its gradient: θ ← θ − η·∇L(θ), where η is the learning rate. Each step reduces L (for small enough η) because the negative gradient is the direction of steepest local decrease. On a convex bowl it converges to the global minimum; it is the workhorse behind training every modern neural network (via backpropagation supplying ∇L). A fail-loud self-check throws unless descending on (x−3)² converges to x=3. ◆ real optimization, node-verified.
Shown on a 1-D convex bowl; real loss surfaces are high-dimensional and non-convex (many minima, saddles), and gradients are estimated on mini-batches (stochastic GD). The step rule and convergence-on-a-bowl are exact.