Interview: 'DQN vs PPO vs SAC vs DPO — which for what?'
mediumAnswer
- DQN: discrete actions, Q-learnable, replay works → Atari, tabular-like.
- PPO: general workhorse, on-policy, LLMs (RLHF), games — reliable + tunable.
- SAC: continuous control, sample-efficient, robotics — max-entropy + off-policy.
- DPO: preference-based LLM alignment — simplest path from preference data.
- Rule: DQN discrete-only; PPO everywhere; SAC continuous + sample-eff; DPO for LLM preferences.
Check yourself — multiple choice
- Same
- DQN: discrete + Atari; PPO: general workhorse, LLM RLHF, games; SAC: continuous control + sample eff (robotics); DPO: LLM preferences (simpler than PPO)
- Random
- Only PPO
DQN discrete / PPO general / SAC continuous / DPO LLM preferences.
#interview
Practise Reinforcement Learning
214 interview questions in this topic.
Related questions
- Interview: 'design an RL system for X' — what's your framework?
- Interview: 'when should you NOT use RL?'
- Interview: 'your DQN agent isn't learning — how do you debug?'
- Interview: 'explain why PPO is on-policy but uses a ratio to importance-sample.'
- Interview: 'walk me through the tradeoffs of SFT-only vs SFT+RLHF vs SFT+DPO.'
- Interview: 'you're given 100 hours of expert data + can simulate — how do you train an agent?'