◄ WORLD V · SONNY 5DART 624 · a helldive back into the mind

THE BEAM SEARCH keep your options open

When a model generates a sequence, taking the single best token at each step (greedy) can paint you into a corner — a great first word can force a bad sentence. Beam search hedges: keep the top-k partial sequences alive at every step, extend them all, and prune back to k. A small amount of lookahead recovers high-probability sequences that greedy throws away. The workhorse decoder of translation and speech.

THE TECHNIQUE keep the top-k partial sequences

The demo shows a case where beam width 2 recovers a better sequence than greedy: live demo







  
HISTORY & CREDIT the decoding heuristic

  
“Bigger beams always give better text.” — for open-ended generation large beams can produce bland, repetitive output; sampling often reads better. cited

  
the trap · greedy commits to the locally-best token and cannot back out.
the hedge · keep the top-k sequences by cumulative score, extend, prune — recover the better path.
heuristic · not optimal (that’s exponential), but a strong, cheap approximation.

  
A little lookahead beats pure greed. decoding






  
RECOMMEND FOR I-13 beam beats greedy, on the compiler

  
On i-13, greedy scores 0.18 (a strong first token, weak finish) but beam-2 finds 0.36 — the better sequence:

  
$ i13 run n2_beam-search.i13
RUN OK · 16 step(s)
greedy = 18   beam_best = 36 (width 2)
beam_wins = 1  recovers_better = 1

  
Recommend as a NULL — a search heuristic (B40). Beam search is a bounded resource choice over a pinned scoring; keeping top-k is amortized cost. NULL — keep your options open.