What are the limits of SHAP values when explaining a model to a stakeholder?
hardAnswer
- SHAP explains the model, not the world: it attributes the model's output, so a spurious feature gets a large attribution if the model relies on it.
- With correlated features, the credit split between them is arbitrary, and swapping two collinear features can move attributions dramatically without changing predictions.
- TreeSHAP is exact for trees but KernelSHAP is an approximation whose variance depends on the sample size.
- And a local attribution is not a causal claim: 'income contributed +0.3' does not mean raising income would change the outcome.
- State that explicitly when a stakeholder wants to act on it.
Check yourself — multiple choice
- SHAP gives causal effects
- It explains the model rather than reality, splits credit arbitrarily among correlated features, and is not a causal statement
- SHAP is always exact
- SHAP replaces validation
SHAP is a model attribution, correlation-sensitive and non-causal.
#interpretability#interview-scenarios
Practise Supervised Learning
215 interview questions in this topic.
Related questions
- How do you justify shipping logistic regression over a boosted ensemble that scores better?
- How do you interpret a coefficient in a multiple linear regression?
- How do you interpret a logistic regression coefficient as an odds ratio?
- How is feature importance computed from a decision tree or random forest?
- What are monotonic constraints in XGBoost / LightGBM and when do you use them?
- You must pick ONE metric for your model. How do you decide?