Bootstrapped DQN — approximate Thompson sampling for deep RL.
hardAnswer
- K parallel Q-heads sharing a body, each trained on a different bootstrap of the replay buffer (mask indicates which head learns from which transition).
- At episode start, sample one head, act greedy w.r.t. it for entire episode → deep exploration (temporally-extended commitment).
- Beats ε-greedy on long-horizon exploration tasks (Deep Sea, hard-exploration Atari).
Check yourself — multiple choice
- Same as DQN
- K Q-heads bootstrapped from different data masks; sample one head per episode, act greedy for whole episode → deep temporal exploration; beats ε-greedy on hard-explore
- Random
- Not real
Bootstrapped DQN: K bootstrapped Q-heads; sample per episode; deep exploration.
#exploration#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.