Filter, wrapper, and embedded feature selection — how do they differ?
mediumAnswer
- Filter: rank features by a statistic — fast, model-agnostic, but ignores feature interactions and doesn't optimize for the downstream model.
- Wrapper: try different feature subsets, retrain the model, pick the best — most accurate but expensive (forward, backward, RFE).
- Embedded: the model itself selects features during training (L1 in linear models, tree importance, or LightGBM's built-in feature filtering).
- Embedded is usually the best cost/quality tradeoff.
Check yourself — multiple choice
- They all do the same thing
- Filter: fast univariate stats; Wrapper: retrain many models; Embedded: model-native (L1, tree importance)
- Wrapper is the fastest
- Filter uses the model's predictions
Filter (stats), Wrapper (train + evaluate subsets), Embedded (built into training). Trade cost vs quality.
#feature-selection#features
Practise Supervised Learning
215 interview questions in this topic.