Reparameterization trick in stochastic policies.
hardAnswer
- For continuous stochastic policy π_θ, sample a = μ + σ ⊙ ε with ε ~ N(0, I) → gradient flows through μ, σ directly.
- Enables low-variance path-wise gradients rather than score-function REINFORCE-style gradients.
- Used in SAC.
- Also foundation of VAEs.
- Doesn't work directly for discrete actions (use Gumbel-softmax as continuous relaxation).
Check yourself — multiple choice
- Same as REINFORCE
- a = μ_θ + σ_θ ⊙ ε with ε ~ N(0, I) → path-wise gradients through μ, σ, much lower variance than REINFORCE; SAC uses it; Gumbel-softmax for discrete
- Random
- Not real
Reparam: a = μ + σε; path-wise low-variance gradients; SAC / VAE.
#policy-methods#deep-rl#theory
Practise Reinforcement Learning
214 interview questions in this topic.
Related questions
- Tabular vs function-approximation RL — the challenge.
- When does tabular Q-learning fail in practice?
- Why is subtracting a baseline OK in policy gradient?
- The log-derivative trick — why is it central to policy gradients?
- What is natural gradient in policy gradient methods?
- DDPG — Deep Deterministic Policy Gradient.