EasyDeepLearn

Quantile regression in QR-DQN — the loss.

hard

Answer

  • Predict N quantiles τi{\tau}_{i} of return distribution.
  • Loss on target y for quantile prediction ZiZ_{i} at level τi{\tau}_{i}: ρ_τ(y  −  Zi)(y\; - \;Z_{i}) where ρ_τ(u) = u(τ - I(u < 0)) (asymmetric absolute error).
  • Total loss  =  Σi\mathrm{loss}\; = \;{\Sigma}_{i} E[ρτi(TDtarget  −  Zi)]E[{\rho}{\tau}_{i}(\mathrm{TD}_{\mathrm{target}}\; - \;Z_{i})].
  • Trains each quantile to correct level.
  • Extension of C51 to continuous quantiles.
  • Foundation of IQN.
Check yourself — multiple choice
  • Random
  • Predict N quantiles ZiZ_{i} at levels τi{\tau}_{i}; loss ρ_τ(y  −  Zi)  =  u(τ  −  I(u<0))(y\; - \;Z_{i})\; = \;u({\tau}\; - \;I(u < 0)) asymmetric absolute error; trains each quantile to correct level; extension of C51 to continuous quantiles
  • Same as MSE
  • Not real

QR loss: ρ_τ(u) = u(τ - I(u<0)); asymmetric absolute error at level τ.

#value-methods#theory

Practise Reinforcement Learning

214 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions