TD3 — Twin Delayed DDPG improvements over DDPG.
hardAnswer
- Fujimoto et al. 2018 fixes: (1) Twin critics: two Q-networks, use min for target → mitigates overestimation.
- (2) Delayed policy updates: update actor every 2 critic steps → gives critic time to catch up.
- (3) Target policy smoothing: add clipped noise to target actions → reduces exploitation of Q's peaks.
- Result: much more stable than DDPG on continuous control.
- Standard baseline before SAC.
Check yourself — multiple choice
- Same as DDPG
- Twin Q (min for target, avoids overestimation) + delayed policy updates + target policy noise smoothing → stable continuous control; SAC/TD3 replaced DDPG
- Random
- Not real
TD3: twin Q + delayed policy + target smoothing → stable continuous control.
#policy-methods#actor-critic#deep-rl
Practise Reinforcement Learning
214 interview questions in this topic.