Why do we prefer simpler models when performance is equal?
easyAnswer
- Occam's razor: among models with similar validation performance, the simpler one usually generalizes better and is cheaper to serve, easier to debug and monitor, and less prone to overfit noise.
- Complexity should be justified by clear gains.
- In practice this drives choices like linear over polynomial when residuals allow, smaller trees over deeper ones, and shallow ensembles over stacks-of-stacks.
Check yourself — multiple choice
- Simpler models are more accurate by definition
- Simpler models generalize better and are easier to maintain when performance ties
- Complex models never overfit
- Occam's razor is about training speed only
Simpler models are preferred at parity because they generalize, serve and maintain better.
#fundamentals#theory
Practise Supervised Learning
215 interview questions in this topic.