When would LDA outperform logistic regression?
hardAnswer
- When (1) classes are approximately Gaussian with similar covariances — LDA is the maximum-likelihood classifier for that model, more efficient than logistic regression which is agnostic to the feature distribution; (2) the training set is small — LDA has fewer parameters and estimates them from all of the data at once; (3) classes are well-separated — logistic regression can suffer from perfect separation, LDA doesn't.
- Logistic regression usually wins when features are non-Gaussian or highly correlated.
Check yourself — multiple choice
- LDA always beats logistic regression
- When class-conditional Gaussianity holds, sample size is small, or classes are well-separated
- LDA only wins on text data
- Never — logistic regression always dominates
LDA is MLE-optimal under its Gaussian assumption and is data-efficient; logistic is more robust otherwise.
#lda-qda#logistic-regression#classification
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Why do we use cross-entropy (log loss) instead of MSE for classification?
- One-vs-Rest, One-vs-One, and softmax — how do you choose for multi-class?
- Write out the softmax function and its main property.
- 'multinomial' vs 'ovr' setting in scikit-learn's LogisticRegression — what changes?
- Precision vs recall — when do you optimize each?
- Why is logistic regression called a linear model if it uses a sigmoid?