What's the difference between probability, odds, and log-odds?
easyAnswer
- Probability p in [0, 1].
- Odds = p / (1 - p) in [0, ∞): 'how many times more likely than not'.
- Log-odds (logit) = log(p / (1 - p)) in (-∞, ∞): a symmetric, unbounded scale, useful for modelling.
- Logistic regression models the log-odds linearly in x, exactly because log-odds live on the real line (matching a linear predictor) and the inverse (sigmoid) automatically bounds probabilities to [0, 1].
Check yourself — multiple choice
- Odds and probability are the same
- Probability ∈ [0,1]; odds = p/(1-p); log-odds = log(p/(1-p)) ∈ (-∞, ∞)
- Log-odds are always positive
- Odds are the same as log-odds
Probability → odds (unbounded above) → log-odds (unbounded, symmetric) = ideal for linear modelling.
#logistic-regression#theory
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- Why not just fit a linear regression for a binary label (linear probability model)?
- Generative vs discriminative classifier — what's the difference?
- What is the curse of dimensionality?
- Parametric vs non-parametric models — what's the difference?
- Why do we prefer simpler models when performance is equal?
- What does the no free lunch theorem say for ML?