Model Predictive Control (MPC) in RL — how does it work?
mediumAnswer
- At each step: (1) roll out learned or hand-crafted model for horizon H, (2) optimize action sequence over that horizon (CEM, LQR, iLQR, gradient descent), (3) execute the first action, (4) re-plan next step.
- Uses: robotics locomotion (PETS, PlaNet), industrial control, quadcopter flight.
- Advantage: adapts to model errors via constant re-planning.
- Weakness: computation per step.
- Combined with learned model for MBRL.
Check yourself — multiple choice
- Same as PPO
- At each step: rollout learned/hand-crafted model for horizon H, optimize action sequence (CEM/iLQR), execute first action, re-plan; adapts to model errors; PETS / PlaNet
- Random
- Not real
MPC: rollout + optimize + execute first + re-plan; PETS / PlaNet MBRL.
#model-based#planning
Practise Reinforcement Learning
214 interview questions in this topic.