Self-play — why does it work for games?
hardAnswer
- Agent plays against copies of itself (past or current) → generates unlimited training data, difficulty auto-scales as agent improves.
- In zero-sum games: converges to Nash equilibrium (Fictitious Self-Play, PSRO).
- AlphaZero: naive self-play.
- AlphaStar: League play (Nash, Main, Exploiters).
- OpenAI Five: pool of frozen past versions.
- Key: variety in opponents prevents collapse to narrow strategy.
Check yourself — multiple choice
- Random
- Play against copies of self → unlimited data, auto-scaling difficulty; converges to Nash in zero-sum; AlphaZero naive self-play; AlphaStar league play; opponent diversity key
- Same as PPO
- Not real
Self-play: unlimited data + auto-difficulty; league play prevents narrow strategy.
#multi-agent
Practise Reinforcement Learning
214 interview questions in this topic.