Categorical vs Gaussian vs beta policies — which and when?
mediumAnswer
- Categorical: discrete actions, softmax over logits.
- Gaussian: continuous, . σ can be state-dependent or state-independent parameter.
- Beta: bounded continuous [a, b] → avoids clipping bias of unbounded Gaussian.
- Tanh-Gaussian (squashed): Gaussian pass through tanh → bounded.
- Modern default: tanh-Gaussian (SAC uses it).
- Beta useful for asymmetric action bounds.
Check yourself — multiple choice
- Only Gaussian
- Categorical (discrete), Gaussian (continuous), Beta (bounded interval, no clip bias), Tanh-Gaussian (SAC default) — squashed to bounded; Beta for asymmetric bounds
- Random
- Not real
Policy: Categorical / Gaussian / Beta / Tanh-Gaussian; SAC uses Tanh-Gaussian.
#policy-methods
Practise Reinforcement Learning
214 interview questions in this topic.