Policy averaging — what is it and why?

medium

Answer

$Take exponentially-weighted average of policy weights over training (Polyak averaging on policy).$
$Reduces variance of policy across training + gives smoother behavior.$
$Used in TD3, SAC.$
$Related concept: SWA (Stochastic Weight Averaging) — averaging final training weights improves generalization.$
$Modern: DreamerV3 averages world model + policy weights.$

Check yourself — multiple choice

Random
Exponential moving average of policy weights (Polyak on policy) → smoother behavior + variance reduction; TD3/SAC target networks are this; DreamerV3 averages world model + policy
Same as SGD
Not real

Policy averaging: Polyak on policy weights; TD3 / SAC / DreamerV3.

#engineering

214 interview questions in this topic.