How is o1-style reasoning safer or riskier?

hard

Answer

$Safer: (1) longer reasoning gives more opportunities to catch errors + reject harmful requests.$
$(2) Explicit deliberation can invoke safety principles.$
$Riskier: (3) can 'think its way' around safety training if not RL-trained on refusal.$
$(4) Chain-of-thought exposes reasoning that could be jailbroken.$
$(5) Increased test-time compute \to some safety mitigations at inference.$
$Modern practice: RL train reasoning models with explicit safety verifier reward.$

Check yourself — multiple choice

Only safer
Safer: catch errors + invoke safety in CoT; Riskier: can reason around safety + CoT exposes jailbreak paths; modern: RL train reasoning with explicit safety verifier reward
Random
Only riskier

Reasoning safety: catches errors but can reason around; RL with safety verifier reward.

#llm#alignment#safety

214 interview questions in this topic.