EasyDeepLearn

Why prefer Jensen-Shannon Divergence over KL for drift?

hard

Answer

  • KL⁡(P∣∣Q)\operatorname{KL}(P \mid \mid Q) is asymmetric + unbounded + undefined when Q=0 for some support.
  • JSD = ½ KL⁡(P∣∣M)\operatorname{KL}(P \mid \mid M) + ½ KL⁡(Q∣∣M)\operatorname{KL}(Q \mid \mid M) where M = (P+Q)/2 is symmetric + bounded to [0, ln 2] (or [0,1] with base 2) + always defined.
  • Interpretable as square of a distance (√JSD is a metric).
  • Bounded scale makes thresholding across features straightforward.
  • Standard choice for drift dashboards.
Check yourself — multiple choice
  • Random
  • KL: asymmetric + unbounded + undefined at Q=0; JSD: symmetric + bounded [0, ln2] + always defined; √JSD is a metric; better for thresholding
  • Same thing
  • KL is better

JSD: symmetric + bounded + always defined; better than KL for drift.

#monitoring#data-quality

Practise MLOps & Data Quality

215 interview questions in this topic.

Read the lessonFlashcardsTake the quiz

Related questions