What can RL do that SFT cannot?
mediumAnswer
- SFT does behavior cloning of ideal responses.
- RL adds: (1) preference between responses (SFT can't compare).
- (2) Learning from FAILED attempts (verifier RL, HER-style relabeling).
- (3) Exploration of the response space beyond demos.
- (4) Direct optimization of test-time desirable behavior (long CoT, tool use, safety).
- (5) Scaling with test-time compute.
- Rule: SFT gives baseline capability; RL adds targeted post-training improvements.
Check yourself — multiple choice
- Same
- Preferences between responses / learning from failures / exploration beyond demos / direct optimization of behavior / test-time scaling — RL adds beyond SFT baseline
- Random
- Only imitation
RL over SFT: preferences + failures + exploration + direct opt + test-time scaling.
#llm#alignment
Practise Reinforcement Learning
214 interview questions in this topic.