R2D2 — distributed recurrent DQN.
hardAnswer
- Kapturowski et al. 2019: recurrent Q-network (LSTM) for partial observability + prioritized replay with sequence-based sampling + distributed actor-learner + burn-in prefix (feed part of sequence with old hidden state before computing loss).
- Combines R (recurrence) + D (distributed) + D (double / dueling).
- Foundational for hard-memory Atari + Agent57.
Check yourself — multiple choice
- Same as DQN
- Recurrent Q (LSTM) + sequence-based prioritized replay + distributed actor-learner + burn-in prefix (initial hidden state); foundation of hard-memory Atari + Agent57
- Random
- Not real
R2D2: recurrent + distributed + prioritized DQN with sequence-based sampling.
#value-methods#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.