EasyDeepLearn

Hard update vs soft (Polyak) update for target networks.

medium

Answer

  • Hard: copy θtarget{\theta}_{\mathrm{target}} ← θ every N steps (DQN: every 10k).
  • Simple.
  • Soft: θtarget{\theta}_{\mathrm{target}} ← τθ + (1-τ) θtarget{\theta}_{\mathrm{target}} every step (SAC/DDPG: τ = 0.005).
  • Smoother, avoids sudden target shifts.
  • Both stabilize TD training.
  • Rule: hard update for large jumps that matter (discrete DQN), soft for continuous / actor-critic settings.
  • Modern default is soft update with τ ~ 0.001-0.01.
Check yourself — multiple choice
  • Only hard
  • Hard: θtarget{\theta}_{\mathrm{target}} ← θ every N steps; Soft (Polyak): θtarget{\theta}_{\mathrm{target}} ← τθ + (1-τ) θtarget{\theta}_{\mathrm{target}} every step (τ ≈ 0.005) — smoother; soft default for actor-critic
  • Random
  • Same thing

Target update: hard every N steps or soft θt{\theta}_{t} ← τθ + (1-τ)θt{\theta}_{t} every step.

#value-methods#engineering

Practise Reinforcement Learning

214 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions