What is a support vector, and why does the SVM only depend on them?
mediumAnswer
- Support vectors are the training points that lie on the margin or inside it (or misclassified for soft margins) — points with non-zero Lagrange multipliers in the dual problem.
- The optimal hyperplane is a linear combination of these support vectors alone: adding or removing non-SV points doesn't change the solution.
- This makes SVMs memory-efficient at inference (store only SVs) and gives an implicit form of feature/data selection.
Check yourself — multiple choice
- All training points contribute to the SVM boundary
- Only the points on/inside the margin — the support vectors — define the classifier
- Support vectors are added at test time
- They come from the test set
Only support vectors have non-zero alpha; the boundary depends on them alone.
#svm#theory
Practise Supervised Learning
215 interview questions in this topic.