Monte Carlo Tree Search — how it works in AlphaGo/MuZero.

hard

Answer

$Iteratively build a tree of possible action sequences.$
$Each iteration: (1) Select — walk from root using UCB1 (or PUCT) balancing exploration + Q.$
$(2) Expand — add new child node.$
$(3) Evaluate — value estimate from neural net (AlphaZero) or rollout.$
$(4) Backup — propagate value up.$
$After N iterations, pick move with highest visit count.$
$(\operatorname{ln}\;N\; / \;n_{a})$
$Foundation of AlphaGo, AlphaZero, MuZero.$

Check yourself — multiple choice

Random
Iteratively Select (UCB / PUCT) + Expand + Evaluate + Backup; pick move by visit count; foundation of AlphaGo / AlphaZero / MuZero
Same as DFS
Not real

MCTS: Select (UCB/PUCT) + Expand + Evaluate + Backup; AlphaZero / MuZero core.

#planning#model-based

214 interview questions in this topic.