What is a policy gradient and why do we use it?
hardAnswer
- A policy gradient directly optimizes the parameters theta of a stochastic policy to maximize expected return: grad .
- Advantages: handles continuous action spaces, learns stochastic policies (crucial for partial observability), and integrates naturally with neural networks.
- Disadvantages: high variance — you need baselines/advantage estimation to stabilize.
Check yourself — multiple choice
- Policy gradients require discrete actions
- Policy gradients optimize policy parameters via grad log pi · advantage
- Policy gradients are only used in tabular settings
- Policy gradients avoid the log-likelihood trick
REINFORCE uses grad log weighted by the advantage.
#policy-methods
Practise Reinforcement Learning
214 interview questions in this topic.