EasyDeepLearn

How do you make Jupyter notebooks reproducible?

medium

Answer

  • Notebooks are anti-reproducible by default: out-of-order execution, hidden state.
  • Fixes: (1) Papermill: parameterize + execute notebooks as scripts programmatically.
  • (2) jupyter nbconvert --execute fresh run.
  • (3) nbdev / nbdime for diff + review.
  • (4) Extract library code into .py; notebook is thin runner.
  • (5) nbstripout git filter to remove outputs from commits.
  • (6) nb-clean on commit.
  • Rule: notebooks for exploration, .py for production.
Check yourself — multiple choice
  • Random
  • Notebooks anti-reproducible by default; fix: Papermill parameterize + execute fresh + nbdev/nbdime diff + extract to .py library + nbstripout / nb-clean strip outputs; notebooks explore, .py produce
  • Just save
  • Not real

Notebook reproducibility: Papermill + fresh execute + .py library + strip outputs.

#reproducibility

Practise MLOps & Data Quality

215 interview questions in this topic.

Related questions