EasyDeepLearn

How do you version prompts and why does it matter as much as model versioning?

medium

Answer

  • Treat a prompt as code: it lives in the repository, changes through review, and every deployed version has an identifier logged with each request.
  • It matters because a prompt edit changes behaviour as much as a retrained model, and a prompt stored in a database and edited by hand produces regressions nobody can trace or revert.
  • Pin the model version alongside it, since the same prompt on a new model version is a different system, and providers update models behind stable aliases.
  • Attach the evaluation results for that prompt version, so promotion is gated on a measured comparison rather than someone's impression.
  • And keep the retrieval configuration in the same versioned unit, because chunking and top-k changes have the same effect as prompt changes.
Check yourself — multiple choice
  • Prompts are configuration, not code
  • Version prompts in the repository with reviews and a logged identifier per request, pinned to a model version and retrieval config, with evaluation results gating promotion
  • Only the model needs versioning
  • Store prompts in a database and edit them live

A prompt change alters behaviour like a retrain, so it needs the same versioning, logging and gating.

#llmops#versioning#reproducibility

Practise MLOps & Data Quality

215 interview questions in this topic.

Related questions