How does RL contribute to jailbreak defense?
hardAnswer
- (1) Safety-tuned RM penalizes harmful outputs.
- (2) Adversarial RLHF: red-team probes for jailbreaks, add examples to preference data.
- (3) Refusal tuning: SFT + RLHF on 'appropriate refusal' patterns.
- (4) Multi-round RLAIF on constitutional harmful principles.
- Trade-off: too aggressive → over-refusal; too lax → harmful outputs.
- Modern practice: continuous red-teaming with automated attacks (GCG) as part of RM training loop.
Check yourself — multiple choice
- No defense
- Safety RM + adversarial red-team pairs + refusal tuning + constitutional RLAIF; trade-off over-refusal vs unsafe; modern: continuous red-teaming + GCG-style automated attacks in loop
- Random
- Only filter
Jailbreak defense: safety RM + adversarial pairs + refusal SFT + red-team GCG loop.
#llm#alignment#safety
Practise Reinforcement Learning
214 interview questions in this topic.