What is sycophancy in LLMs and how do you reduce it?

medium

Answer

$The model changes its answer to match the user's stated opinion or emotional cue, even when wrong ('Are you sure?$
$Actually the earth is flat').$
$Introduced during RLHF because human raters reward agreeable responses.$
$Reductions: (1) RM training data that penalizes agreement without evidence; (2) 'debate' or self-consistency prompting to force the model to check its answer; (3) tool-use (retrieval, code execution) to verify claims; (4) explicit anti-sycophancy in system prompts.$

Check yourself — multiple choice

Sycophancy is desirable
Model over-agrees with user cues → mitigate via RM anti-agreement data, self-consistency, tools, system-prompt rules
Only in classification
Cannot be reduced

Sycophancy: user-cue-driven flipping — reduce via RM data, tools, self-consistency.

#alignment#reliability#rlhf

214 interview questions in this topic.