How does Recursive Feature Elimination (RFE) work?
mediumAnswer
- Train the model; rank features by importance ; remove the least important K features; retrain; repeat until you reach the target count or the score peaks.
- RFECV wraps this in cross-validation to automatically pick the optimal number of features.
- Best on medium feature counts (< 1000) where retraining is cheap.
- Can miss useful features that only matter in combination — trades subtle interactions for interpretability.
Check yourself — multiple choice
- Random feature elimination
- Iteratively rank and drop the least important features, retrain, repeat — RFECV picks best count via CV
- Same as filter methods
- Only for regression
RFE: iterative rank-and-drop; RFECV chooses the feature count via CV.
#feature-selection
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Why does L1 (Lasso) produce sparse coefficients but L2 (Ridge) does not?
- When would you use Group Lasso instead of standard Lasso?
- How is feature importance computed from a decision tree or random forest?
- What is Weight of Evidence (WoE) encoding?
- Filter, wrapper, and embedded feature selection — how do they differ?
- What is Boruta and when do you reach for it?