How does selection bias affect ML systems?

hard

Answer

$Training data doesn't represent deployment population.$
$Examples: (1) recommender only trained on items shown \to can't learn about unshown items (exposure bias).$
$(2) credit scoring: trained on approved applicants only \to censored.$
$(3) medical diagnosis: only diagnosed patients labeled.$
$Fixes: (1) importance weighting toward deployment distribution.$
$(2) inverse propensity scoring (IPS) for exposure bias.$
$(3) exploration / random rollout to break selection.$
$(4) counterfactual evaluation.$
$(5) actively collect from underrepresented segments.$

Check yourself — multiple choice

Random
Training data unrepresentative of deployment (exposure bias / censoring); fix via importance weight + IPS + exploration/random rollout + counterfactual eval + targeted collection
Same as overfit
Not real

Selection bias: IPS + exploration + counterfactual + targeted collection.

#data-quality#monitoring

215 interview questions in this topic.