What is 'honesty' as an alignment target and how do you train for it?
mediumAnswer
- Honesty means the model expresses calibrated uncertainty and refuses to fabricate.
- Training levers: (1) SFT examples that model 'I don't know' correctly (harder than it looks — must not over-refuse); (2) preference data that rewards refusing over confabulating on unknown facts; (3) calibration losses that align the model's stated confidence with observed accuracy; (4) tool use (retrieval, code execution) to verify before answering.
Check yourself — multiple choice
- Honesty is unrelated to hallucinations
- Calibrated uncertainty + refusal of unknown facts; train via SFT/preference/calibration + tools
- Only regex
- Same as filtering
Honesty: calibrated uncertainty + tool use; alignment target beyond helpfulness/harmlessness.
#alignment#hallucinations#reliability
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- Why do LLMs hallucinate and how do you reduce hallucinations?
- What is sycophancy in LLMs and how do you reduce it?
- Name three types of LLM hallucination.
- How do you detect hallucinations in production?
- How well-calibrated are LLM confidences and how do you fix them?
- What is supervised fine-tuning (SFT) and where does it fit in the alignment pipeline?