CEM — Cross-Entropy Method for planning.
mediumAnswer
- Sample-based optimization: (1) sample K action sequences from Gaussian, (2) evaluate returns, (3) select elite (top 10%), (4) refit Gaussian to elites, (5) repeat.
- Simple, embarrassingly parallel, no gradients.
- Used in model-predictive control (PETS, model-based RL) and QT-Opt (Google robotics).
- Slower than gradient-based but more robust to non-differentiable models / rewards.
Check yourself — multiple choice
- Random
- Sample K action seqs from Gaussian, evaluate, refit to elite top 10%, repeat; no gradients; used in MPC + QT-Opt; robust to non-diff models
- Same as PPO
- Not real
CEM: sample-elite-refit iteration; no gradients; MPC / QT-Opt default.
#planning#exploration
Practise Reinforcement Learning
214 interview questions in this topic.