What is schema drift and how do you detect it?

medium

Answer

$Schema drift = upstream data changes shape: column added / removed / renamed, dtype changes (int \to string), enum values expanded.$
$Silent failures because pipelines don't crash but predictions become garbage.$
$Detect via schema validation on ingestion (Great Expectations, TFDV, Pandera): assert expected columns + types + allowed values.$
$Fail pipeline loudly on mismatch; never fill with defaults silently.$
$Modern practice: contract-based data pipelines where producers must declare schema changes.$

Check yourself — multiple choice

Same as data drift
Upstream data changes shape (columns/dtypes/enums); detect via schema validation on ingestion (GE / TFDV / Pandera); fail loudly; contract-based pipelines
Not a problem
Random

Schema drift: shape changes; validate on ingestion; fail loudly.

#data-quality#monitoring

215 interview questions in this topic.