How do you validate data quality in an ML pipeline?
mediumAnswer
- Set up automated checks with tools like Great Expectations, TFDV, or Deequ.
- Validate: schema (types, columns present), value ranges, allowed categories, null rates, uniqueness, referential integrity, and distributional stats (mean, std, quantiles) vs a reference.
- Fail loudly on violations, quarantine bad rows, and page on-call for critical breaches.
Check yourself — multiple choice
- Data validation should be skipped in production
- Schema, ranges, and distributional checks catch most upstream issues
- Great Expectations only validates images
- Null rates are not worth monitoring
Automated schema + distribution + range checks catch most issues.
#data-quality
Practise MLOps & Data Quality
215 interview questions in this topic.