Hard-margin vs soft-margin SVM — what's the difference?
mediumAnswer
- Hard-margin SVM finds the hyperplane that separates classes with the largest margin — only works when classes are linearly separable, otherwise there is no feasible solution.
- Soft-margin adds slack variables ≥ 0 that let some points violate the margin (xi in the margin, xi > 1 misclassified) and penalizes them via C * sum xi.
- C is a regularization knob: large C → few violations, high variance; small C → many violations, high bias.
- Almost every real SVM is soft-margin.
Check yourself — multiple choice
- Hard-margin is always used in practice
- Soft-margin allows margin violations penalized by C — the only realistic option for noisy / overlapping data
- Both are identical when C = 0
- Hard-margin uses kernels; soft-margin doesn't
Soft-margin = slack variables + C penalty. Hard-margin only works for perfectly separable data.
#svm
Practise Supervised Learning
215 interview questions in this topic.