Learn · Beam Search vs Greedy
Beam Search vs Greedy Decoding
A language model gives you a probability for every possible next token. How you turn those probabilities into a sentence matters. Greedy decoding grabs the single best token at each step; beam search keeps several candidates alive and often finds a more probable sentence overall.
Prompt:
The weather today is ___
The next-token probabilities below are illustrative — a small, fixed toy model conditioned only on the previous word. The decoding maths (log-prob scoring, argmax, beam pruning) is real.
How many partial sequences beam search keeps alive at each step. B = 1 is identical to greedy decoding. Higher B explores more, at more compute.
Greedy vs best beam
Greedy decoding
Always picks the argmax token — independent of B.
Best beam
The beam search tree
Each step expands every kept beam into its possible continuations, re-ranks all candidates by cumulative log-probability, and keeps the top B. Kept sequences are highlighted; pruned ones are dimmed.
Why it works
Local vs global
Greedy is short-sighted: it commits to the best token now, even if a slightly worse first token opens up much more probable continuations. Beam search delays that commitment, so it can find a higher-probability sequence overall.
Scoring in log space
A sequence's probability is the product of its per-token probabilities. Multiplying many
small numbers underflows, so we sum log-probs instead
(Math.log). Less-negative totals are more probable; ranking is identical.
The downside
The most probable text is often the most boring. Beam search tends to produce safe, bland, sometimes repetitive output, which is why creative generation usually prefers sampling (temperature / top-p) over pure beam search.
This is a toy: a real model conditions each prediction on the entire context, not just the previous word, and has a vocabulary of tens of thousands of tokens. Production decoding also adds length normalisation and repetition penalties. For related demos, see Temperature & Sampling or the Learn page.