What are the key tricks that make DQN work?
hardAnswer
- (1) Experience replay buffer: store transitions and sample randomly to break correlations and improve data efficiency.
- (2) Target network: use a delayed copy of Q for the TD target, updated slowly, to stabilize training.
- (3) Reward clipping and frame stacking (for Atari).
- (4) Double DQN reduces overestimation bias.
- (5) Prioritized replay samples surprising transitions more often.
- (6) Dueling architecture separates value and advantage.
Check yourself — multiple choice
- Experience replay is only used for on-policy methods
- A target network stabilizes TD-target estimates
- Double DQN increases overestimation
- DQN cannot use CNN inputs
A slowly updated target network prevents the moving-target problem.
#value-methods#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.