THE TEACHER FORCING train on the truth, test on yourself
To train a sequence model to predict the next token, you could feed it its own previous prediction — but early on those are garbage and errors compound. Teacher forcing feeds the real previous token instead, so every step gets a clean target and training is fast and stable. The catch is exposure bias: at inference the model must feed itself, a distribution it never saw in training — and small errors snowball.
THE TECHNIQUE feed the true previous token, not the guess
The demo shows each step’s input is the ground-truth previous token — errors do not compound during training: live demo
HISTORY & CREDIT Williams & Zipser · 1989
“Teacher forcing has no downside.” — it causes exposure bias; the model never learns to recover from its own mistakes (scheduled sampling and RL fixes address it). cited
the clean signal · feed the true previous token, so each step’s target is correct — fast, stable training. the catch · exposure bias — at test the model feeds itself, a distribution it never trained on. 1989 · Williams & Zipser.
A clean classroom, a messy exam. training
RECOMMEND FOR I-13 the ground truth, on the compiler
On i-13, step 2’s input is the true token 1, step 3’s is the true 2 — the model’s own guesses never feed back in training:
$ i13 run n2_teacher-forcing.i13
RUN OK · 22 step(s)
input_step2 = 1 input_step3 = 2 (ground truth)
no_error_compound = 1 exposure_bias = 1
Recommend as a NULL — a training protocol (B39). Feeding the ground-truth input is a fixed choice; exposure bias is its known cost. A policy, not a same-function DOF. NULL — train on the truth, test on yourself.