What is inductive bias and why does every model have one?
mediumAnswer
- Inductive bias is the set of assumptions a learning algorithm uses to generalize from finite data to unseen examples.
- Without an inductive bias, any function consistent with the training data is equally likely — no learning is possible.
- Examples: linear regression assumes a linear conditional mean; CNNs assume locality and translation equivariance; trees assume axis-aligned splits; k-NN assumes locality in feature space.
- Choosing a model = choosing an inductive bias that matches your data.
Check yourself — multiple choice
- Inductive bias is a bug to eliminate
- It's the set of assumptions that lets a model generalize beyond the training data
- Only deep nets have inductive biases
- Inductive bias only matters for classification
Inductive bias = generalization assumptions. Without it, no learning happens.
#theory#fundamentals
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Parametric vs non-parametric models — what's the difference?
- Why do we prefer simpler models when performance is equal?
- What does the no free lunch theorem say for ML?
- What is the bias-variance tradeoff?
- Generative vs discriminative classifier — what's the difference?
- What is the curse of dimensionality?