Why does DQN stack 4 consecutive frames?
easyAnswer
- Atari observation is a single frame — insufficient for Markov state (need velocity: ball direction, projectile motion).
- Stack 4 last frames as input → agent can infer motion.
- Alternative: recurrent Q-net (R2D2) learns temporal features directly.
- Also: reward and terminal from previous step included.
- Standard preprocessing: grayscale, downscale to 84×84, frame-skip 4 (act every 4 frames).
- Foundation of DQN Atari benchmark.
Check yourself — multiple choice
- Random
- Single Atari frame is not Markovian (need velocity); stack 4 frames → agent infers motion; alt: recurrent Q-net; standard preprocessing incl. grayscale + 84×84 + frameskip 4
- Same as batch
- Not needed
Frame stack: recover Markov state from single frames; canonical DQN preprocessing.
#deep-rl#engineering
Practise Reinforcement Learning
214 interview questions in this topic.