What is the generalization gap and how do you shrink it?
mediumAnswer
- Generalization gap = expected test loss minus training loss.
- A large gap means overfitting.
- Shrink it with more data, stronger regularization (L2, dropout, weight decay), data augmentation, simpler models, ensembling, or early stopping.
- It relates to the model's effective capacity and the sample size — the classic result is that gap decreases like sqrt(capacity / n).
Check yourself — multiple choice
- It is the difference between two random validation folds
- It is test loss minus training loss; large gap ⇒ overfitting
- It disappears with more parameters
- It is minimized by removing regularization
Generalization gap = test - train loss. Shrink it with data, regularization, or simpler models.
#evaluation#regularization#theory
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- How do you detect and fix overfitting?
- What's the Bayesian interpretation of Ridge regression?
- What is the bias-variance tradeoff?
- L1 vs L2 regularization — what's the difference?
- What is k-fold cross-validation and when do you use stratified or grouped folds?
- Generative vs discriminative classifier — what's the difference?