RL for reasoning (o1, DeepSeek-R1) — the paradigm shift.
hardAnswer
- Train LLM to generate long chain-of-thought reasoning by RL with automatic reward from verifier (math answer correct, code passes tests).
- Model learns to think longer, retry, self-correct.
- DeepSeek-R1: pure RL from base model (no SFT step) → emergent reasoning behaviors.
- Uses GRPO (group-normalized rewards).
- Test-time compute becomes a knob.
- Marks shift from imitation-based to verification-based post-training.
Check yourself — multiple choice
- Random
- RL with automatic verifier reward (math correct / code passes) → model learns to think longer, retry, self-correct; DeepSeek-R1 pure RL from base; GRPO; shift from imitation to verification
- Same as RLHF
- Not real
RL reasoning: verifier reward + long CoT; DeepSeek-R1 GRPO; verification paradigm.
#llm#alignment
Practise Reinforcement Learning
214 interview questions in this topic.