THE TEMPERATURE SAMPLING the dial between safe and wild
To generate text, sample from the model’s distribution — but first divide the logits by a temperature T before the softmax. Low T (→0) sharpens toward the single most likely token: safe, repetitive. High T flattens toward uniform: creative, risky. T=1 is the model’s honest distribution. One knob turns the same network from a cautious autocomplete into a wild improviser.
THE TECHNIQUE softmax(logits / T)
The demo softmaxes [1,2,3] at low vs high temperature — low sharpens the top, high flattens it: live demo
HISTORY & CREDIT the softmax temperature
“Higher temperature makes the model smarter/more creative.” — it only makes sampling flatter; too high and it degenerates into noise. cited
the scale · divide logits by T before softmax. the dial · T→0 → argmax (safe); T large → uniform (wild); T=1 the true distribution. generation · the single knob behind “creative” vs “precise” sampling.
One knob, from cautious to wild. sampling
RECOMMEND FOR I-13 the sharpening, on the compiler
On i-13, softmax([1,2,3]) at low T puts 0.87 on the top token; at high T only 0.51 — the dial:
$ i13 run n2_temperature-sampling.i13
RUN OK · 5078 step(s) · call depth 31
p(top) low-T = 0.867 (sharp)
p(top) high-T = 0.506 (flat) low_sharpens = 1
Recommend as a NULL — a pinned reshape + entropy (B39/B47). The tempered distribution is a fixed function of T; the actual token drawn is random (B47). NULL — the dial between safe and wild.