What is a model feedback loop and why is it dangerous?
hardAnswer
- Model's predictions influence future training data → self-reinforcing.
- Example: (1) recommender pushes popular items → users click popular → more training signal for popular → runaway effect.
- (2) hiring algorithm reject certain profile → no future data for that profile.
- (3) fraud model detects one pattern → attackers shift, but model doesn't learn new pattern from its own denials.
- Detect via causal reasoning + exposure logs.
- Fix via exploration / random baseline traffic + counterfactual evaluation + off-policy correction.
Check yourself — multiple choice
- Random
- Predictions shape future training data → self-reinforcing (recsys popularity spiral / hiring exclusion / fraud pattern lock-in); fix via exploration + counterfactual + off-policy correction
- Only positive
- Not real
Feedback loop: self-reinforcing bias; explore + counterfactual + off-policy fix.
#monitoring#safety
Practise MLOps & Data Quality
215 interview questions in this topic.