EasyDeepLearn

What do you actually test in a CI pipeline for a model?

hard

Answer

  • Split the tests by what they protect.
  • Data tests validate schema, ranges, null rates and cardinality on the incoming data, and they should fail the pipeline, because training on broken data is worse than not training.
  • Pipeline tests run the whole path on a tiny fixture to catch shape and type errors in seconds.
  • Behavioural tests assert properties rather than accuracy: that a known-obvious example is classified correctly, that a monotonic relationship holds, that a perturbation which should not matter does not change the prediction.
  • A performance gate compares the candidate against the current production model on a frozen evaluation set, with a tolerance rather than an exact number.
  • And a serving test loads the artefact and scores a request, which catches the dependency mismatch that breaks deployments.
Check yourself — multiple choice
  • Only unit tests on utility functions
  • Data validation, a fast end-to-end pipeline run on a fixture, behavioural property tests, a performance gate against production on a frozen set, and an artefact-loading serving test
  • Only final accuracy
  • Models cannot be tested in CI

Data, pipeline, behaviour, performance gating and serving each catch a distinct class of failure.

#mlops#reproducibility

Practise MLOps & Data Quality

215 interview questions in this topic.

Related questions