EasyDeepLearn

All 215 Supervised Learning interview questions

Classification, regression, evaluation and the classic bias-variance interview questions.

Every question below opens on its own page, with a full answer, the formulas written out and a multiple-choice check. Free to read, no account needed.

LessonFlashcardsQuizMock

Fundamentals & theory

  1. What is the bias-variance tradeoff?easy
  2. How do you detect and fix overfitting?easy
  3. What is k-fold cross-validation and when do you use stratified or grouped folds?medium
  4. Generative vs discriminative classifier — what's the difference?medium
  5. What is the curse of dimensionality?medium
  6. Parametric vs non-parametric models — what's the difference?easy
  7. Why do we prefer simpler models when performance is equal?easy
  8. What does the no free lunch theorem say for ML?medium
  9. In one sentence, what is PAC learning?hard
  10. What is VC dimension and why should you care?hard
  11. What is Empirical Risk Minimization?medium
  12. Why do we care whether the loss landscape is convex?medium
  13. What are the roles of the training, validation and test sets?easy
  14. What does the IID assumption mean and when is it violated?medium
  15. Why is the base rate of the positive class critical to know?medium
  16. What is inductive bias and why does every model have one?medium
  17. What is the generalization gap and how do you shrink it?medium
  18. Why does OLS have a closed-form solution but logistic regression doesn't?medium
  19. What's the Bayesian interpretation of Ridge regression?hard
  20. What is a Generalized Linear Model and when is it the right tool?medium
  21. Why do we use cross-entropy (log loss) instead of MSE for classification?medium
  22. How is logistic regression fit in practice?medium
  23. Your scikit-learn logistic regression warns 'lbfgs failed to converge'. What do you do?medium
  24. How does Bayes' theorem drive Naive Bayes classification?easy
  25. The 'naive' independence assumption is almost always false. Why does Naive Bayes still work?medium
  26. Why do we compute Naive Bayes scores in log space?easy
  27. What are the core assumptions behind Linear Discriminant Analysis (LDA)?medium
  28. Why does KNN degrade badly in high dimensions?medium
  29. How do KD-trees and ball-trees speed up KNN, and when do they stop helping?medium
  30. Why not just fit a linear regression for a binary label (linear probability model)?medium
  31. What's the difference between probability, odds, and log-odds?easy
  32. Why is a single decision tree considered a 'high-variance' model?medium
  33. Why do decision trees (and boosted trees) fail to extrapolate?medium
  34. What is the out-of-bag (OOB) score in Random Forests?medium
  35. What is a support vector, and why does the SVM only depend on them?medium
  36. Why don't kernel SVMs scale well to millions of samples?medium
  37. What is 'deviance' and why is it used to evaluate GLMs?hard
  38. You must pick ONE metric for your model. How do you decide?medium
  39. An interviewer asks you to build a churn model. What do you do before touching any algorithm?easy

Linear & regularized regression

  1. L1 vs L2 regularization — what's the difference?easy
  2. Why is logistic regression called a linear model if it uses a sigmoid?medium
  3. What are the classical assumptions of linear regression?medium
  4. How do you interpret a coefficient in a multiple linear regression?easy
  5. What is VIF and when do you worry about multicollinearity?medium
  6. How do you handle strong multicollinearity in a linear model?medium
  7. How do you detect and fix heteroscedasticity?hard
  8. Adding a feature raised R^2. Does that mean the model improved?easy
  9. Can R^2 be negative? What does that mean?medium
  10. When would you use robust regression (Huber, RANSAC) instead of OLS?medium
  11. What's the main risk of polynomial regression?easy
  12. When should you add interaction features to a linear model?medium
  13. When does log-transforming the target help a regression model?medium
  14. What do Box-Cox and Yeo-Johnson transforms do?medium
  15. Why does Ridge regression give more stable coefficients than OLS?medium
  16. Why does L1 (Lasso) produce sparse coefficients but L2 (Ridge) does not?medium
  17. When is Elastic Net better than pure Lasso or Ridge?medium
  18. How do you choose the regularization strength (lambda / alpha)?medium
  19. Why must you standardize features before applying L1 or L2 regularization?easy
  20. In one sentence, what does the LARS algorithm compute?hard
  21. When would you use Group Lasso instead of standard Lasso?hard
  22. When is quantile regression more useful than mean (OLS) regression?hard
  23. What is 'perfect separation' in logistic regression and how do you fix it?hard
  24. L1 vs L2 in logistic regression — practical differences.easy
  25. What is Laplace (additive) smoothing in Naive Bayes?easy
  26. Pre-pruning vs post-pruning in decision trees — what's the difference and when do you use each?medium
  27. How does cost-complexity (CCP) pruning work?hard
  28. How does early stopping work in gradient boosting and why is it important?easy
  29. How does the C parameter in an SVM affect the fit?medium
  30. How does SVM adapt to regression (SVR)?medium
  31. What is Huber loss and why is it a compromise between MAE and MSE?medium
  32. What does the quantile (pinball) loss measure?hard
  33. What is stability selection?hard
  34. What are monotonic constraints in gradient boosting and when are they worth using?hard
  35. When would you use quantile regression instead of predicting the mean?medium

Logistic regression & classification basics

  1. Precision vs recall — when do you optimize each?easy
  2. How do you read a confusion matrix and derive the key metrics?easy
  3. One-vs-Rest, One-vs-One, and softmax — how do you choose for multi-class?medium
  4. Write out the softmax function and its main property.easy
  5. How do you interpret a logistic regression coefficient as an odds ratio?medium
  6. How do class weights work in logistic regression, and when do you use them?medium
  7. 'multinomial' vs 'ovr' setting in scikit-learn's LogisticRegression — what changes?medium
  8. Gaussian NB vs Multinomial NB vs Bernoulli NB — when do you use each?medium
  9. When would LDA outperform logistic regression?hard
  10. What is the nearest centroid (Rocchio) classifier and when is it useful?easy
  11. How does temperature scaling calibrate a classifier's probabilities?medium
  12. Why is accuracy a bad primary metric for many real-world problems?easy
  13. What is F-beta score and when is F1 not enough?medium
  14. Macro vs micro vs weighted averaging in multi-class metrics — how do you choose?medium
  15. What is Cohen's kappa and why is it useful?medium
  16. What is the Matthews Correlation Coefficient (MCC) and when should you use it?medium
  17. What is log loss (cross-entropy) and what does a specific value mean?medium
  18. What is the Brier score and how does it compare to log loss?medium
  19. What is top-k accuracy and when do you use it?easy
  20. What is balanced accuracy?easy
  21. What is the geometric mean (G-mean) in classification metrics?medium
  22. What is Youden's J statistic and how is it used?medium
  23. How do you interpret an ROC curve, and what point matters?medium
  24. What does ROC-AUC really measure, and what's a good value?easy
  25. What are the main pitfalls of relying on ROC-AUC?medium
  26. How do you select the decision threshold for a binary classifier?medium
  27. How do you evaluate a multi-label classifier?medium
  28. MAP, NDCG, MRR — when do you use each in ranking?hard
  29. How do you evaluate a classifier when FP and FN have different costs?hard

Naive Bayes, LDA & KNN

  1. Why is Naive Bayes still a solid baseline for text classification?easy
  2. QDA vs LDA — how do you choose between them?medium
  3. How is LDA used for supervised dimensionality reduction?medium
  4. How do you choose k in k-Nearest Neighbours?easy
  5. How do you choose the distance metric for KNN?medium
  6. What's the point of distance-weighted KNN?easy
  7. You trained Naive Bayes on balanced data but deploy on data where positives are 1%. What happens and what do you do?hard
  8. How does KNN imputation work and when is it a good choice?medium

Decision trees & random forests

  1. Random forest vs gradient boosting — which do you pick and why?medium
  2. Gini impurity, entropy, and classification error — which do trees actually use?medium
  3. How do decision trees handle categorical features?medium
  4. What are the most impactful hyperparameters of a decision tree?easy
  5. How do decision trees handle missing values?medium
  6. What criterion does a regression tree use to choose splits?easy
  7. How is feature importance computed from a decision tree or random forest?medium
  8. How does bagging reduce variance?medium
  9. Why does Random Forest sample features at each split, not just once per tree?medium
  10. How do Extra Trees differ from Random Forests?medium
  11. Which Random Forest hyperparameters actually matter for tuning?medium
  12. Are Random Forest probability estimates well-calibrated?medium
  13. What is Boruta and when do you reach for it?hard

Gradient boosting

  1. In one sentence, what is Gradient Boosting?medium
  2. What is the role of the learning rate (shrinkage) in GBM?easy
  3. What did XGBoost bring on top of vanilla GBM?hard
  4. What are LightGBM's key innovations vs XGBoost?hard
  5. What makes CatBoost different from XGBoost / LightGBM?hard
  6. Which XGBoost hyperparameters have the biggest impact, and in what tuning order?medium
  7. How do you handle class imbalance in XGBoost / LightGBM?medium
  8. How do XGBoost, LightGBM, and CatBoost each handle categorical features?medium
  9. How does gradient boosting handle missing values without imputation?medium
  10. What are monotonic constraints in XGBoost / LightGBM and when do you use them?hard
  11. Rule of thumb: when do you pick XGBoost vs LightGBM vs CatBoost?medium
  12. Is adding more features always safe with gradient boosting?medium

SVMs & kernels

  1. Why do SVMs use kernels?medium
  2. Hard-margin vs soft-margin SVM — what's the difference?medium
  3. What does gamma control in an RBF-kernel SVM?medium
  4. How do SVMs handle multi-class problems?medium
  5. Why is feature scaling critical for SVMs?easy

Ensembling & stacking

  1. How does stacking work and when does it help?hard
  2. Soft voting vs hard voting — which is usually better and why?easy
  3. Why is diversity between base models more important than their individual accuracy in an ensemble?medium

Classification metrics

  1. When should you use PR-AUC instead of ROC-AUC?medium
  2. How do you check whether a classifier's probabilities are well-calibrated?medium
  3. Platt scaling vs isotonic regression for probability calibration — how do you choose?medium
  4. How do you choose the decision threshold for a classifier in a business setting?medium
  5. When do you need calibrated probabilities rather than a good ranking?medium
  6. Your positive class is 0.5%. Is resampling your first move?medium

Regression metrics

  1. RMSE vs MAE — how do you pick?easy
  2. What are the pitfalls of MAPE (Mean Absolute Percentage Error)?medium
  3. What is sMAPE and why is it used?medium
  4. What is explained variance score and how does it differ from R^2?hard
  5. When would you use MSLE (mean squared log error) instead of MSE?hard
  6. Is R^2 a useful metric for non-linear models (RF, GBM, neural nets)?medium

Cross-validation & data splitting

  1. Leave-One-Out CV — when is it a good idea, and when is it a bad idea?medium
  2. What is nested cross-validation and when do you need it?hard
  3. Why would you use repeated k-fold instead of standard k-fold?medium
  4. How do you set up cross-validation for time series?hard
  5. How do you pick the train/validation/test split sizes?easy
  6. How does cross-validation change for heavily imbalanced classification?medium
  7. You use stratified k-fold on a dataset with duplicate customer records — why is it wrong?medium
  8. When would you use the bootstrap for model evaluation instead of k-fold?hard
  9. When is cross-validation NOT necessary?easy
  10. When does a random train/test split give you a misleading score?medium

Feature engineering & encoding

  1. Which models need feature scaling and which don't?easy
  2. What is target leakage and how do you prevent it?medium
  3. Ordinal vs nominal encoding — how do you choose?easy
  4. What is target encoding and when is it useful?medium
  5. How do you prevent leakage in target encoding?hard
  6. What is frequency (count) encoding?easy
  7. What is feature hashing (the 'hashing trick') and when is it useful?medium
  8. What is Weight of Evidence (WoE) encoding?hard
  9. How do you encode cyclical features like hour-of-day or day-of-week?medium
  10. How do you turn text into features for a classical model?easy
  11. You have a feature with 50,000 unique category values. How do you encode it?medium
  12. When should you manually engineer interaction features?medium
  13. When is discretization (binning) a useful feature transformation?medium
  14. StandardScaler vs MinMaxScaler vs RobustScaler — how do you choose?easy
  15. Filter, wrapper, and embedded feature selection — how do they differ?medium
  16. How exactly does SMOTE generate synthetic minority-class samples?medium
  17. What does ColumnTransformer do and when do you need it?easy
  18. You did StandardScaler().fit_transform(X) then split into train/test. Why is this wrong?easy
  19. Why is target encoding dangerous, and how do you do it safely?hard

Feature selection

  1. How does Recursive Feature Elimination (RFE) work?medium
  2. How does permutation importance work and when is it better than impurity-based importance?medium
  3. Can you use SHAP values for feature selection?medium
  4. When can aggressive feature selection actually hurt performance?medium
  5. What does VarianceThreshold do and when is it useful?easy
  6. Mutual information vs chi-square vs ANOVA F for feature selection — how do you pick?medium

Imbalanced data

  1. How do you handle class imbalance in a dataset?medium
  2. SMOTE, Borderline-SMOTE, SVM-SMOTE, ADASYN — when do you use each?hard
  3. Random undersampling vs Tomek links vs cluster-centroid undersampling — what's each for?medium
  4. What is focal loss and why does it help with class imbalance?hard
  5. Class weights vs resampling for imbalance — which do you reach for first?easy

Missing data & outliers

  1. MCAR, MAR, MNAR — what are these and why do they matter?medium
  2. What is iterative (MICE) imputation?hard
  3. IQR, Z-score, Isolation Forest, LOF — how do you pick an outlier detection method?medium

Hyperparameter tuning

  1. Grid search vs random search for hyperparameter tuning — which do you use?easy
  2. How does Bayesian hyperparameter optimization work at a high level?hard
  3. What is Optuna's TPE sampler and why is it popular?hard
  4. How do Successive Halving / Hyperband / ASHA speed up hyperparameter search?hard
  5. How do you tune when you have multiple objectives (accuracy AND latency AND size)?hard

Pipelines & leakage

  1. Why should preprocessing live inside a scikit-learn Pipeline rather than being applied manually before?easy
  2. Your new model has 60% AUC and the baseline had 75%. How do you debug?medium
  3. You detect concept drift in production. What do you do?medium
  4. How do you train and evaluate a model when labels arrive weeks after predictions?hard
  5. Your model has great offline metrics. What do you check before serving it in production?medium
  6. Your model scores AUC 0.92 offline but barely helps in production. What are the usual causes?hard
  7. How do you handle noisy labels in a supervised dataset?hard
  8. How do you decide how often to retrain a supervised model?medium

Interview scenarios

  1. Name three situations where you should NOT reach for machine learning.medium
  2. What are the limits of SHAP values when explaining a model to a stakeholder?hard
  3. How do you justify shipping logistic regression over a boosted ensemble that scores better?medium

Other topics