Dyna — early model-based RL framework.
mediumAnswer
- Sutton 1990: interleave (1) real interaction with env, (2) model updates from real data, (3) planning: multiple simulated Q-learning updates from the learned model.
- Rebuilds Q from cheap simulated experience.
- Cornerstone of model-based RL.
- Modern deep versions: Dyna-Q with NN model, MBPO (Janner et al. 2019).
- Advantage: model-based sample efficiency + model-free performance.
Check yourself — multiple choice
- Random
- Interleave real interaction + model update + planning via simulated Q-updates from learned model; cornerstone of MBRL; modern: MBPO / Dyna-Q with NN
- Same as DQN
- Not real
Dyna: real + model + planning simulated updates; MBRL foundation.
#model-based#planning
Practise Reinforcement Learning
214 interview questions in this topic.