AlphaZero — key ideas.
hardAnswer
- (1) MCTS as policy-improvement operator: search improves the raw NN prior.
- (2) Self-play generates training data: play against your current model.
- (3) Single neural net outputs BOTH policy prior and value estimate.
- (4) Train NN to predict MCTS output (targets: MCTS action distribution + game outcome).
- (5) No handcrafted features.
- Iterate → superhuman Go, Chess, Shogi in 24h.
- Modern extension: MuZero learns dynamics model too.
Check yourself — multiple choice
- Random
- MCTS + self-play + single NN (policy prior + value) + train to match MCTS output; superhuman Go/Chess/Shogi in 24h; MuZero adds learned dynamics
- Same as DQN
- Not real
AlphaZero: MCTS + self-play + policy-value NN; superhuman game play.
#planning#model-based
Practise Reinforcement Learning
214 interview questions in this topic.
Related questions
- Monte Carlo Tree Search — how it works in AlphaGo/MuZero.
- Dyna — early model-based RL framework.
- World models (Ha & Schmidhuber, Dreamer) — big idea.
- MuZero — model-based RL without knowing the env dynamics.
- Model Predictive Control (MPC) in RL — how does it work?
- CEM — Cross-Entropy Method for planning.