When does chain-of-thought HURT performance?

hard

Answer

$(1) Well-known factual retrieval — 'What is the capital of France?' — CoT adds noise and error paths without any benefit.$
$(2) Simple pattern completion / one-step arithmetic.$
$(3) Tasks where the model has been RLHF-tuned to answer directly and CoT triggers style drift.$
$(4) Recent work (Sprague 2024) showed CoT gains are concentrated in math / logic / symbolic tasks — on many NLP tasks, CoT is neutral or slightly hurts.$
$Rule: use CoT for multi-step / verifiable tasks; not for retrieval / classification.$

Check yourself — multiple choice

CoT always helps
CoT hurts on single-step retrieval / classification; concentrated gain is in math / logic / symbolic (Sprague 2024)
CoT only for pretraining
CoT and self-consistency are the same

CoT: shines on multi-step / verifiable; hurts or is neutral on retrieval / classification.

#prompting#chain-of-thought

214 interview questions in this topic.