Go-Explore — how does it solve hard-exploration?
hardAnswer
- Ecoffet et al. 2019, 2021: (1) archive all visited states (cell-hashed).
- (2) 'Go' phase: teleport back to promising archived state (or replay actions).
- (3) 'Explore' phase: random exploration from that state; if new state discovered, add to archive.
- (4) Later: distill trajectories into robust policy via imitation learning.
- Broke Montezuma's Revenge + Pitfall.
- Insight: separate remembering promising states from acting from them.
Check yourself — multiple choice
- Random
- Archive visited states → teleport back to promising ones + random-explore from there + distill into robust policy; solved Montezuma + Pitfall by separating 'remember' from 'explore'
- Same as RND
- Not real
Go-Explore: archive-teleport-explore-distill; solved Montezuma / Pitfall.
#exploration#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.