How do you monitor performance when labels are delayed?

hard

Answer

$Ground truth arrives days / weeks / months later (loan default, customer churn).$
$Strategies: (1) proxy metrics available immediately (clicks for CTR, purchase within 1h).$
$(2) short-window versions of long-term label (7-day approx of 90-day churn).$
$(3) leading indicators from user behavior.$
$(4) partial evaluation on subset with fast feedback.$
$(5) rolling window backfill: compute metric as labels arrive.$
$(6) Alert on distribution changes even without labels.$
$Match business KPI once available.$

Check yourself — multiple choice

Wait
Proxy metrics (clicks/purchases) + short-window approximation + leading indicators + partial fast-feedback subset + rolling backfill + distribution alert without labels; match business KPI when arrives
Random
Not real

Delayed labels: proxy + short-window + leading + backfill + drift alert.

#monitoring

215 interview questions in this topic.