How does beam search work and what is its main failure mode?
mediumAnswer
- At each decoding step, keep the k highest-probability sequences ('beams') so far, expand each by one token, keep the top k of the resulting k*|.
- Approximate search — better than greedy, cheaper than exhaustive.
- Failure mode: length bias (shorter sequences get higher joint probability) — fix with length normalization .
- Also produces 'safe, boring' outputs on open-ended generation — LLM decoders now prefer sampling with temperature / top-p.
Check yourself — multiple choice
- Beam search is exhaustive
- Keep top-k beams at each step; failure: length bias → length normalization; boring outputs on open-ended tasks
- Same as greedy decoding
- Only for CNNs
Beam search: top-k pruned search; length bias → length norm; boring outputs → temperature / top-p.
#transformers#rnn
Practise Deep Learning
214 interview questions in this topic.