Why do we compute Naive Bayes scores in log space?
easyAnswer
- The product can be astronomically small (underflow) when there are many features (e.g., thousands of words in a document).
- Taking logs turns the product into a sum: log log .
- Sums are numerically stable, and the argmax is preserved because log is monotonic.
- All practical NB implementations work in log space.
Check yourself — multiple choice
- To make training slower
- Products underflow with many features; log-sums are numerically stable and preserve argmax
- To make features Gaussian
- So we can use gradient descent
Log-space avoids underflow when multiplying many small probabilities.
#naive-bayes#optimization
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- What is Empirical Risk Minimization?
- Why do we care whether the loss landscape is convex?
- Why does OLS have a closed-form solution but logistic regression doesn't?
- Why do we use cross-entropy (log loss) instead of MSE for classification?
- How is logistic regression fit in practice?
- Your scikit-learn logistic regression warns 'lbfgs failed to converge'. What do you do?