Why is Montezuma's Revenge such a famous benchmark?
mediumAnswer
- Extreme sparse reward: dozens of steps needed before any positive reward.
- DQN + ε-greedy scores 0.
- Requires deep, temporally-extended exploration — random policies never find rewards.
- Solved sequentially by intrinsic-motivation methods: pseudo-counts → RND → Go-Explore → Agent57 → NGU.
- Became the canonical stress-test for exploration algorithms, similar to CIFAR-10 for image classification.
Check yourself — multiple choice
- Random
- Extreme sparse reward → random policy scores 0; canonical exploration benchmark; solved sequentially by pseudo-counts → RND → Go-Explore → Agent57 → NGU
- Same as Pong
- Not real
Montezuma's Revenge: extreme sparse reward; canonical hard-exploration benchmark.
#exploration
Practise Reinforcement Learning
214 interview questions in this topic.