What is VC dimension and why should you care?
hardAnswer
- The Vapnik-Chervonenkis dimension of a hypothesis class is the largest number of points it can shatter (label with any possible +/- assignment).
- Higher VC dimension = more expressive class = higher risk of overfitting on small samples.
- Linear classifiers in d dimensions have VC dim d+1.
- VC bounds justify why you need more data as model capacity grows, and why regularization limits effective VC dim.
Check yourself — multiple choice
- VC dimension measures training accuracy
- It is the largest set size the class can shatter, capturing capacity
- It equals the number of parameters
- It only applies to SVMs
VC dim = shattering capacity. Higher VC dim → needs more data to generalize.
#theory
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Generative vs discriminative classifier — what's the difference?
- What is the curse of dimensionality?
- Parametric vs non-parametric models — what's the difference?
- Why do we prefer simpler models when performance is equal?
- What does the no free lunch theorem say for ML?
- In one sentence, what is PAC learning?