How does top-k differ from top-p sampling?
easyAnswer
- Top-k: sample from the k highest-probability tokens each step, ignoring the rest.
- Fixed cardinality regardless of the distribution's shape.
- Top-p (nucleus): sample from the smallest set of tokens whose cumulative .
- Adapts to the distribution — takes fewer tokens when the model is confident, more when it's uncertain.
- Top-p is usually preferred as it maintains diversity where useful and precision where confident.
- Combining top-k=50 + top-p=0.95 is a common safety net.
Check yourself — multiple choice
- Top-k adapts to distribution shape
- Top-k: fixed k tokens; top-p: smallest set with cumulative — adapts to shape (preferred)
- Same behavior
- Only top-k works
Top-k fixed size; top-p adaptive to distribution — top-p usually preferred.
#generation#decoding#sampling
Practise LLMs & GenAI
214 interview questions in this topic.