THE GRU — a dart

THE GRU the LSTM, simplified

The gated recurrent unit strips the LSTM to two gates and no separate cell state: an update gate z that decides how much of the past to keep, and a reset gate that decides how much to forget when proposing a new state. hᵗ = (1−z)·hᵗ₋₁ + z·h̃. Fewer parameters, often the same accuracy — a leaner memory that trains a little faster.

THE TECHNIQUE hᵗ = (1−z)·hᵗ₋₁ + z·h̃

The demo shows the update gate z control the blend: z=0 keeps the past, z=1 fully updates: live demo

HISTORY & CREDIT Cho et al. · 2014

“GRU is strictly worse than LSTM because it is simpler.” — on many tasks GRU matches or beats LSTM with fewer parameters; neither dominates. cited

two gates · update z (keep vs replace) and reset (forget when proposing).
the blend · hᵗ = (1−z)hᵗ₋₁ + z·h̃ — z=0 keep, z=1 replace.
2014 · Cho et al. — the LSTM, leaner.

The same idea, fewer moving parts. recurrence

RECOMMEND FOR I-13 the gate, on the compiler

On i-13, update gate z=0 keeps h=100; z=1 replaces it with the new value 5 — two gates vs the LSTM’s three:

$ i13 run n2_gru.i13 RUN OK · 53 step(s) z=0: keep 100 z=1: update to 5 z_controls = 1 gates = 2

Recommend as a NULL — a gated recurrence (B39). The blend is a pinned function of the gate; a fixed structure. NULL — the LSTM, simplified.