THE CHAIN RULE — a dart

THE CHAIN RULE backprop's whole secret

How does a network know how to change a weight buried ten layers deep? The chain rule: the derivative of a composition is the product of the derivatives — (f∘g)′ = f′(g(x))·g′(x). Backpropagation is nothing more than the chain rule applied backward through the layers, multiplying local gradients from the loss to every weight. Leibniz’s 300-year-old calculus is the engine of every deep net.

THE TECHNIQUE (f∘g)′ = f′(g(x))·g′(x)

The demo differentiates (2x+1)² both by the chain rule and directly — they agree: live demo

HISTORY & CREDIT Leibniz · backprop

“Backpropagation is a new deep-learning invention.” — it is the chain rule (Leibniz), reverse-mode autodiff; applied to nets by Werbos 1974, popularized by Rumelhart-Hinton-Williams 1986. cited

compose · a net is functions stacked: loss(…(layer(x))).
multiply backward · each layer’s local derivative multiplies the gradient flowing back — the chain rule.
Leibniz · the calculus; reverse-mode automatic differentiation is its efficient form.

A gradient, threaded back through every layer. calculus

RECOMMEND FOR I-13 the product, on the compiler

On i-13, d/dx (2x+1)² at x=1 is 12 by the chain rule (2·3·2) and 12 directly (8x+4):

$ i13 run nn_chain-rule.i13 RUN OK · 37 step(s) chain = 12 -- f'(g(x))*g'(x) = 2*(2*1+1)*2 direct = 12 -- 8x+4 chain_ok = 1

Recommend as a NULL — an identity (B39). The chain rule is exact calculus; every correct differentiator agrees. Backprop is its bookkeeping, not a same-function difference. NULL — backprop’s whole secret.