MBPO — Model-Based Policy Optimization.
hardAnswer
- Janner et al. 2019: use an ensemble of forward models.
- Every real step, generate k short imagined rollouts (branch factor: many rollouts, short length ~ 1-15 steps).
- Train off-policy actor-critic (SAC) on mixture of real + imagined.
- Ensembles give uncertainty → keep rollout short where model unreliable.
- Reaches model-free performance with 10-100× fewer real interactions.
- Key MBRL milestone.
Check yourself — multiple choice
- Same as SAC
- Ensemble of forward models + short (1-15 step) imagined rollouts branched from each real step → SAC on real + imagined mixture; 10-100× fewer real steps than model-free
- Random
- Not real
MBPO: ensemble forward models + short imagined rollouts + SAC on mix.
#model-based#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.