What are monotonic constraints in gradient boosting and when are they worth using?
hardAnswer
- They force the predicted output to move in one direction as a feature increases.
- Useful when domain knowledge is certain: risk should not fall as debt rises, and price should not fall as square footage rises.
- Benefits are regulatory defensibility, more stable behaviour in sparse regions of the feature space, and mild regularization that often helps on small data.
- Cost is bias if the true relationship is not monotone, so check that the constraint does not degrade validation performance.
- Supported directly in XGBoost, LightGBM and CatBoost.
Check yourself — multiple choice
- They speed up training
- They force the prediction to move one way with a feature — good for defensibility and stability, biased if the truth is not monotone
- They only work in linear models
- They remove the need for validation
Monotonic constraints encode certain domain knowledge, trading potential bias for stability.
#gradient-boosting#regularization
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- How does early stopping work in gradient boosting and why is it important?
- How do you detect and fix overfitting?
- L1 vs L2 regularization — what's the difference?
- What is the generalization gap and how do you shrink it?
- How do you handle strong multicollinearity in a linear model?
- What's the main risk of polynomial regression?