World models (Ha & Schmidhuber, Dreamer) — big idea.
hardAnswer
- Learn a compact latent dynamics model of the environment: encoder + RNN dynamics + reward predictor.
- Train policy inside the model ('imagination') rather than the real env.
- Massive sample efficiency: real env only needed to keep the model calibrated.
- Dreamer V3: SOTA on many benchmarks with modest data.
- Foundation of modern model-based deep RL.
- Also enables offline planning by rolling out counterfactuals.
Check yourself — multiple choice
- Random
- Learn latent dynamics + reward predictor; train policy inside imagination (learned model); massive sample efficiency; Dreamer V3 SOTA on many benchmarks with modest data
- Same as PPO
- Not real
World models: latent dynamics + imagined rollouts; Dreamer V3 SOTA MBRL.
#model-based#planning#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.