The S-curve that turns any real number into a value between 0 and 1: σ(x) = 1/(1+e⁻ˣ). It gave early neural nets a smooth, differentiable switch and reads naturally as a probability. But its derivative peaks at just ¼ (at x=0) and vanishes toward the tails — stack many and the gradient dies, the reason deep sigmoid nets were nearly untrainable before ReLU.
THE TECHNIQUE σ(x) = 1/(1+e⁻ˣ)
The demo computes σ(0)=½ and the derivative σ(1−σ), max ¼ (i13 has no exp — it runs a Taylor series): live demo
HISTORY & CREDIT the logistic function
“Sigmoid is the natural activation for hidden layers.” — its ≤¼ gradient makes deep nets vanish; modern hidden layers use ReLU, sigmoid mostly for output gates. cited
the squash · map (−∞,∞) → (0,1), smoothly. the flaw · derivative σ(1−σ) ≤ ¼, near 0 at the tails — gradients vanish in depth. logistic · Verhulst’s curve; the classic neural activation.
A smooth switch that dies in the deep. activation
RECOMMEND FOR I-13 the derivative, on the compiler
On i-13 (exp via Taylor), σ(0)=0.5, σ(2)=0.88, and the max derivative is exactly 0.25:
$ i13 run nn_sigmoid.i13
RUN OK · 1969 step(s) · call depth 33
s0 = 0.5 s2 = 0.8808
ds0 = 0.25 -- the max slope (vanishing culprit)
half_at_zero = 1 saturates = 1
Recommend as a NULL — a pinned function (B39). σ and its derivative are fixed; the vanishing is an analytic property, not a same-function DOF. NULL — squash to a probability.