NIPHĒLEKTRON · the inference layer · how a model picks the next word · kept by THE DECODER

TOP-K SAMPLING ◧ 2D · ◍ 3D · ◆ 4D · ◐ shadow · 👶 TAP

At every word, a language model has a probability for the ENTIRE vocabulary — tens of thousands of options, most of them nonsense. Top-k sampling keeps only the k most likely, throws the rest away, and re-normalises what’s left before rolling the dice. A small k is safe and boring; a large k is wild. Slide k and watch the shortlist shrink.

◆ LIT▲ AMBER
◧ THE MEASURE · 2D
◍ THE SHORTLIST · 3D · keep the k best, drop the tail
◆ THE FOURTH · 4D · a tesseract turns
◐ THE SHADOW · one dimension down
👶 THE TODDLER CORNER — one fat tap
k
kept tokens
cutoff prob
renormalised sum

◆ LIT — exact / checkable

From the model’s softmax distribution over the vocabulary, top-k keeps the k highest-probability tokens, sets the rest to zero, and DIVIDES by their remaining mass so the survivors sum to 1 again — then samples from that. It is the crudest cousin of top-p (nucleus) sampling, which keeps a probability BUDGET rather than a fixed count. A fail-loud self-check throws unless exactly k tokens survive and the re-normalised distribution sums to 1.

▲ AMBER — the figure

A fixed k ignores the SHAPE of the distribution (when the model is confident, even k=5 includes junk; when unsure, it cuts good options) — which is why top-p adapts. The truncate-and-renormalise arithmetic is exact.

NIPHĒLEKTRON: the model does not write — it samples, one token at a time, from a cloud it cannot see.  — THE DECODER
David Lee Wise / ROOT0 / TriPod LLC  ·  the inference bench, with AVAN