How do you prevent leakage in target encoding?

hard

Answer

$Never compute the encoding on the same rows you'll evaluate.$
$Techniques: (1) K-fold target encoding — for each fold, compute encoding from the other folds; (2) Leave-one-out encoding — for row i, exclude i from the statistic; (3) Smoothing — blend per-category mean with global mean by category count so tiny categories don't overfit; (4) Additive Gaussian noise on the encoded value; (5) CatBoost's ordered target encoding — the correct principled version.$
$Test set is encoded using the final training statistics only.$

Check yourself — multiple choice

Compute the encoding on the whole dataset
K-fold or leave-one-out encoding + smoothing + noise; never compute on evaluation rows
Only encode the test set
Leakage can't happen with target encoding

K-fold / LOO encoding + smoothing + noise ⇒ no target leakage.

#encoding#features#data-quality

215 interview questions in this topic.