How do you handle 99.9% normal / 0.1% anomaly training data?
hardAnswer
- (1) Semi-supervised: train only on normal (assume clean).
- (2) Isolation Forest / LOF: designed for exactly this ratio.
- (3) If you have SOME labeled anomalies (>50), oversample them + SMOTE + supervised (imbalanced XGBoost).
- (4) PU learning if unlabeled is 'mostly normal'.
- (5) Cost-sensitive: assign high FN cost.
- Avoid: naive balanced accuracy, single-threshold optimization — use PR AUC.
Check yourself — multiple choice
- Ignore imbalance
- Semi-sup on normals / Isolation Forest / LOF / PU learning / cost-sensitive; if any labels → SMOTE + imbalanced XGBoost; PR AUC not accuracy
- Random
- Only oversample
99.9% normal: semi-sup / iForest / LOF / PU / cost-sensitive; PR AUC.
#anomaly-detection#applications
Practise Unsupervised Learning
214 interview questions in this topic.