What is Constitutional AI?
hardAnswer
- Anthropic's alignment framework (Bai et al., 2022): the model is given a list of principles ('the constitution' — e.g., 'be helpful and harmless') and asked to critique + rewrite its own outputs when they violate these principles.
- The rewrites become preference pairs used to train a reward model or run RLAIF.
- Reduces human labeling burden and makes safety principles explicit / auditable.
- Foundation of Claude's training.
Check yourself — multiple choice
- Manual annotation for every case
- Model critiques + rewrites its outputs against explicit principles → preference pairs for RLAIF → auditable alignment
- Only for math
- No self-critique
Constitutional AI: self-critique against explicit principles → RLAIF alignment.
#alignment#safety
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- How is 'refusal' behavior trained into LLMs?
- How do you balance helpfulness vs harmlessness in RLHF?
- What is an instruction hierarchy in modern LLMs?
- What are the main LLM safety and alignment concerns?
- What is supervised fine-tuning (SFT) and where does it fit in the alignment pipeline?
- How is high-quality instruction-tuning data constructed?