How does self-consistency improve CoT?
mediumAnswer
- Sample K CoT traces at higher temperature (T=0.7-1.0), extract the final answer from each, and majority-vote (or weighted vote).
- Robust to sporadic reasoning errors: even if 3/10 traces are wrong, the correct answer wins the majority.
- Cost: K× more tokens generated.
- Gains: another 5-15% on math benchmarks like GSM8K on top of CoT (Wang et al., 2022).
- Standard in evaluation harnesses for reasoning benchmarks.
Check yourself — multiple choice
- Take the mean logit
- Sample K CoT traces at higher T, majority-vote the final answer → 5-15% gains on math over single-CoT
- Same as beam search
- Only for classification
Self-consistency: sample K CoTs + majority vote → robust to sporadic errors.
#prompting#chain-of-thought#generation
Practise LLMs & GenAI
214 interview questions in this topic.