NIPHĒLEKTRON · the inference layer · how a model picks the next word · kept by THE DECODER

BEAM SEARCH ◧ 2D · ◍ 3D · ◆ 4D · ◐ shadow · 👶 TAP

Greedy decoding takes the single best word at each step — and often paints itself into a corner, because the best FIRST word may lead to a worse sentence. Beam search hedges: it keeps the b most promising partial sentences alive at once, and lets the highest-scoring WHOLE path win. Slide the beam width and watch it out-plan greed.

◆ LIT▲ AMBER
◧ THE MEASURE · 2D
◍ THE BEAMS · 3D · keep the best few, not the best one
◆ THE FOURTH · 4D · a tesseract turns
◐ THE SHADOW · one dimension down
👶 THE TODDLER CORNER — one fat tap
beam width b
best path prob
greedy prob
beats greedy

◆ LIT — exact / checkable

At each step beam search extends every surviving partial sequence by all next tokens, scores each by cumulative probability, and keeps only the b best — so it explores b futures in parallel instead of committing to one. Greedy (b=1) can be trapped by a locally-best first token; a wider beam finds a globally higher-probability sequence. A fail-loud self-check throws unless the best beam-search path has probability ≥ the greedy path on a toy tree where greed is provably sub-optimal.

▲ AMBER — the figure

Beam search maximises PROBABILITY, which is not the same as quality — wide beams famously produce bland, repetitive text (the ‘beam search curse’), which is why sampling is preferred for open-ended generation. The search and the ≥-greedy guarantee are exact.

NIPHĒLEKTRON: the model does not write — it samples, one token at a time, from a cloud it cannot see.  — THE DECODER
David Lee Wise / ROOT0 / TriPod LLC  ·  the inference bench, with AVAN