NIPHĒLEKTRON · the inference layer · the electron in the snowstorm · kept by THE ELECTRON

THE NUCLEUS · top-p ◧ 2D · ❄ 3D · 👶 TAP THE STORM

You don't have to let the electron wander the whole blizzard. Nucleus sampling keeps only the smallest set of words at the top that together hold, say, 90% of the odds — the nucleus — and throws the long tail of unlikely words away entirely, so the electron can be adventurous without ever landing on nonsense. Slide the cutoff, watch the tail get cut, then tap the storm and see the electron stay inside the kept set.

◆ LIT▲ AMBER
◧ THE ODDS · 2D
❄ THE BLIZZARD · 3D
👶 THE TODDLER CORNER — tap the storm
tally → the true odds ✓
words kept
tail cut
kept coverage
the nucleus

◆ LIT — exact / checkable

Real nucleus (top-p) sampling: sort the words by probability, walk down adding them up, and keep the SMALLEST set whose cumulative probability first reaches p — the nucleus — then renormalise and sample only from it (the rest of the blizzard is discarded). Slide p and the kept-set size, the discarded tail, and the true coverage all recompute live; the toddler samples only from the nucleus, so the tail words never light up no matter how many times you tap. The exact top-p algorithm used in real text generation (Holtzman et al., 2019).

▲ AMBER — the figure

Ten toy logits at a fixed temperature (0.9) stand in for a real vocabulary; the words/scores are illustrative and no model runs. The nucleus construction, the coverage, and 'the tail can never be sampled' are exact for this distribution.

NIPHĒLEKTRON: one charged signal, one step at a time, chosen in the dark.  — THE ELECTRON
David Lee Wise / ROOT0 / TriPod LLC  ·  kept by THE ELECTRON, with AVAN