Why don't chat LLMs use beam search?
mediumAnswer
- Beam search maximizes joint probability but produces 'safe, boring' outputs — highest-probability sequences are generic and repetitive.
- Human raters strongly prefer sampled outputs (with T + top-p) in open-ended generation.
- Beam search still shines in constrained-output tasks (structured NER, MT) where the objective is exact-match precision.
- RLHF and DPO further shift the optimal decoding distribution away from mode-seeking beam search.
Check yourself — multiple choice
- Beam is standard for chat
- Beam produces boring, repetitive high-prob outputs; open-ended chat wants sampling with T + top-p
- Beam is faster
- Beam is more expressive
Beam: safe/boring outputs; sampling wins for open-ended chat.
#generation#decoding
Practise LLMs & GenAI
214 interview questions in this topic.