SAC — Soft Actor-Critic mechanism and advantages.

hard

Answer

$E[{\Sigma}\;{\gamma}^{t}\;(r_{t}\; + \;{\alpha}\;H({\pi}( \cdot \mid s_{t})))]$
$Adds entropy bonus \to exploration + robustness.$
$Off-policy, stochastic policy (reparameterized Gaussian), twin Q-critics (TD3-style).$
$Automatically tunes α to hit target entropy.$
$Robust across many continuous-control tasks with default hyperparams.$
$Modern default for continuous action deep RL (Haarnoja 2018).$

Check yourself — multiple choice

Random
Max-entropy RL: max $E[{\Sigma}\;{\gamma}^{t}\;(r\; + \;{\alpha}\;H({\pi}))]$ ; off-policy stochastic reparam Gaussian policy + twin Q + auto α; robust default for continuous control
Same as TD3
Not real

SAC: max-entropy off-policy AC; twin Q + auto α; continuous-control default.

#policy-methods#actor-critic#deep-rl

Practise Reinforcement Learning

214 interview questions in this topic.

Read the lesson Flashcards Take the quiz

Related questions