How do you detect and fix overfitting?
easyAnswer
- Detect it when training loss keeps dropping but validation loss rises, or when train accuracy is much higher than validation.
- Fixes: more data or augmentation, regularization (L1/L2, dropout), simpler model, early stopping, cross-validation, and ensembling.
- In deep learning, add dropout, weight decay, and monitor the validation curve.
Check yourself — multiple choice
- Increase model capacity
- Add regularization or more data
- Train longer without validation
- Remove the validation set
Regularization, more/better data, and early stopping are the standard fixes.
#evaluation#regularization
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- What is the generalization gap and how do you shrink it?
- What is the bias-variance tradeoff?
- L1 vs L2 regularization — what's the difference?
- What is k-fold cross-validation and when do you use stratified or grouped folds?
- What are the roles of the training, validation and test sets?
- How do you handle strong multicollinearity in a linear model?