Reward clipping in DQN — trade-offs.
mediumAnswer
- DQN Atari clips rewards to {-1, 0, +1} → stabilizes Q-learning across games with vastly different reward scales (Ms. Pac-Man vs Skiing).
- Cost: agent doesn't distinguish 1-point vs 100-point reward → suboptimal for scale-sensitive tasks.
- Modern replacement: reward normalization by running SD (used in PPO), or distributional RL that models absolute return distribution.
- Best of both worlds: keep scale info but normalize for gradients.
Check yourself — multiple choice
- Always needed
- Clip to {-1, 0, +1} stabilizes across scales but loses magnitude info; modern replacement: reward normalization by running SD (PPO) or distributional RL absolute returns
- Random
- Not needed
Reward clip: stable but lossy; PPO uses running-SD reward normalization.
#deep-rl#engineering
Practise Reinforcement Learning
214 interview questions in this topic.