Dueling DQN — architecture.

hard

Answer

$Q(s, \;a)\; = \;V(s)\; + \;A(s, \;a)\; - \;\mathrm{mean}_{a}$
$Improves learning when many actions have similar values (V dominates); doesn't waste capacity redundantly encoding V per action.$
$Combined with Double DQN + prioritized replay + noisy nets \to Rainbow DQN (Hessel et al. 2018).$

Check yourself — multiple choice

Same as DQN
Split Q into $V(s)\; + \;A(s, \;a)\; - \;\mathrm{mean}_{a}$ A; better when many actions have similar Q; part of Rainbow DQN
Random
Not real

Dueling: Q = V + A - mean A; efficient when actions similarly valued.

#value-methods#deep-rl

214 interview questions in this topic.