Why add an entropy bonus to the policy objective?
mediumAnswer
- Encourages stochastic exploration + prevents premature convergence to suboptimal deterministic policy.
- Loss = -E[log π · A] - β H(π).
- Also gives robustness (multiple near-optimal actions).
- Foundation of SAC (with automatic α tuning) and A3C-style entropy regularization.
- Trade-off: too high β → agent won't commit; too low → early exploitation.
- In LLM RLHF: entropy prevents mode collapse toward one response.
Check yourself — multiple choice
- Random
- Encourages stochastic exploration + prevents premature convergence to deterministic policy; foundation of SAC (auto α); in RLHF prevents mode collapse
- Same as noise
- Not needed
Entropy bonus: stochastic exploration + robustness; SAC / RLHF default.
#policy-methods#exploration
Practise Reinforcement Learning
214 interview questions in this topic.