Value decomposition (VDN, QMIX) — for multi-agent Q-learning.

hard

Answer

$Q(s, \;a_{1}, \;, \;a_{n})$
$Q_{\mathrm{total}}\; = \;{\Sigma}$
$Q_{\mathrm{total}}\; = \;f_{\mathrm{mix}}(Q_{1}, \;, \;Q_{n}\;s)$
$Foundation of centralized-training-decentralized-execution paradigm for cooperative multi-agent RL.$

Check yourself — multiple choice

Random
VDN: $Q_{\mathrm{total}}\; = \;{\Sigma}$ $Q_{i}(s, \;a_{i})$ — additive; QMIX: monotonic mixing $( \partial f / \partial Q_{i}\; \ge \;0)$ — richer; both enable centralized training + decentralized execution
Same as DQN
Not real

VDN (additive) / QMIX (monotonic mixing): CTDE for cooperative multi-agent Q.

#value-methods#multi-agent

214 interview questions in this topic.