Mutual information vs chi-square vs ANOVA F for feature selection — how do you pick?
mediumAnswer
- Mutual information: measures general (non-linear, non-monotonic) dependency between feature and target — the most powerful filter, works for both categorical and continuous.
- Chi-square: for categorical features + categorical target; tests independence in a contingency table.
- ANOVA F: for continuous features + categorical target; tests whether class means differ.
- Use MI as a default; for categorical-categorical; ANOVA F for numeric-categorical when you assume linear-ish relations and want speed.
Check yourself — multiple choice
- Chi-square is for continuous features only
- MI captures non-linear dependencies (general); for categorical-categorical; ANOVA F for numeric-categorical linear separation
- All three are identical
- ANOVA F is the same as MI
MI = general, , ANOVA F = numeric-categorical linear separation.
#feature-selection
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Why does L1 (Lasso) produce sparse coefficients but L2 (Ridge) does not?
- When would you use Group Lasso instead of standard Lasso?
- How is feature importance computed from a decision tree or random forest?
- What is Weight of Evidence (WoE) encoding?
- Filter, wrapper, and embedded feature selection — how do they differ?
- How does Recursive Feature Elimination (RFE) work?