How do you choose k in k-Nearest Neighbours?
easyAnswer
- Cross-validation over odd values (avoids ties in binary problems).
- Small k (1-3) = low bias, high variance — sensitive to noise.
- Large k = smooth boundary, higher bias, lower variance.
- A common heuristic is k ≈ sqrt(n).
- Weighted KNN (weights inversely proportional to distance) reduces sensitivity to k.
- For classification with C classes, k should not be a multiple of C (to break ties).
Check yourself — multiple choice
- Always k = 1
- Cross-validate over odd k; small k = variance, large k = bias; ~sqrt(n) as a heuristic
- k must equal n
- k must be even
CV over odd k. Small k = high variance, large k = high bias.
#knn#hyperparameter-tuning
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- How do you choose the regularization strength (lambda / alpha)?
- How do you choose the distance metric for KNN?
- What's the point of distance-weighted KNN?
- Why does KNN degrade badly in high dimensions?
- How do KD-trees and ball-trees speed up KNN, and when do they stop helping?
- What is the nearest centroid (Rocchio) classifier and when is it useful?