THE LABEL SMOOTHING — a dart

THE LABEL SMOOTHING never be 100% sure

Training on hard one-hot labels teaches a model to be infinitely confident — it pushes the correct logit toward +∞ and overfits. Label smoothing softens the target: give the true class 1−ε and spread ε across the rest. The model learns to leave a little probability for other answers, which calibrates its confidence and generalizes better. A one-line humility.

THE TECHNIQUE y = (1−ε)·onehot + ε/K

The demo smooths a one-hot label with ε=0.1 over K=10 classes — true class 0.91, others 0.01, sums to 1: live demo

HISTORY & CREDIT Szegedy et al. · 2016

“Label smoothing always improves accuracy.” — it improves calibration and often accuracy, but can hurt when you later distill the model (it erases useful logit structure). cited

the problem · one-hot targets push the correct logit to infinity — overconfidence.
the fix · true class 1−ε, the rest share ε/K — calibrated confidence.
2016 · Szegedy et al. (Inception-v3).

A little doubt, deliberately trained in. regularization

RECOMMEND FOR I-13 the softened target, on the compiler

On i-13, ε=0.1 over K=10: true class 0.91, each other 0.01, summing to 1:

$ i13 run n2_label-smoothing.i13 RUN OK · 26 step(s) true_class = 0.91 other = 0.01 (x9) total = 1.00 less_confident = 1

Recommend as a NULL — a pinned target transform (B39). The smoothed label is a fixed function of ε and K. NULL — never be 100% sure.