What is target encoding and when is it useful?
mediumAnswer
- Replace each category value with a statistic of the target for that category — typically the mean of y within the category (mean encoding).
- Great for high-cardinality categoricals (zip codes, product SKUs, user IDs) where one-hot creates thousands of columns.
- Adds signal from the label into the feature.
- Big risk: leakage.
- Always compute the encoding on the training fold only and apply to validation, or use K-fold target encoding, smoothing (blend with global mean by count), and noise injection.
- CatBoost's ordered boosting bakes this in.
Check yourself — multiple choice
- Encode categories as their frequency
- Replace category with the mean of y in that category — great for high cardinality; watch for leakage
- Encode as one-hot
- Encode by alphabetical order
Target encoding = per-category y-statistic. Powerful for high-cardinality; needs CV/smoothing to avoid leakage.
#encoding#features#feature-engineering
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Ordinal vs nominal encoding — how do you choose?
- How do you encode cyclical features like hour-of-day or day-of-week?
- How do you turn text into features for a classical model?
- You have a feature with 50,000 unique category values. How do you encode it?
- When should you add interaction features to a linear model?
- How do decision trees handle categorical features?