How do you handle high-cardinality categorical features?
mediumAnswer
- One-hot explodes memory; label encoding assumes ordinal (bad).
- Better: (1) Target encoding with proper CV to avoid leakage.
- (2) Hashing trick (fixed-size, collision-tolerant).
- (3) Embedding layer (deep models, ~sqrt(cardinality) dims).
- (4) Frequency encoding.
- (5) Group rare categories to 'other' by frequency threshold.
- (6) For trees: LightGBM native categorical + CatBoost ordered boosting.
- Beware new categories at serve time — need fallback.
Check yourself — multiple choice
- One-hot always
- Target encoding (CV-safe) / hashing trick / embedding layer / frequency encoding / group rare / tree-native categorical (LightGBM/CatBoost); handle new categories at serve
- Random
- Drop it
High cardinality: target encoding / hashing / embedding / group rare.
#features#pipeline
Practise MLOps & Data Quality
215 interview questions in this topic.
Related questions
- How do you prove there is no training-serving skew?
- What is point-in-time correctness, and how does violating it look in practice?
- What are common sources of data leakage in an ML pipeline?
- What problem does a feature store solve?
- When and how do you retrain a production model?
- How do you handle missing values in a production pipeline?