What breaks when you change the embedding model in a live RAG system?
hardAnswer
- Everything in the index, because embeddings from different models are not comparable: vectors live in different spaces, so mixing them silently produces nonsense similarities.
- You must re-embed the entire corpus and rebuild the index, which costs money and time proportional to the corpus.
- Dimensionality often changes, so the store's schema and any dimensionality assumptions change too.
- Retrieval quality can shift in both directions, so you need your retrieval evaluation set before the swap to compare.
- The safe pattern is to build the new index alongside the old one, compare on the eval set, then cut over atomically.
Check yourself — multiple choice
- Nothing, embeddings are interchangeable
- Vectors from different models are incomparable: re-embed the whole corpus, handle dimension changes, and build the new index in parallel to compare before cutting over
- Only latency changes
- Just update the query encoder
Different embedding spaces cannot be mixed, so a full re-index and parallel comparison are required.
#embeddings#vector-db#rag
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- What's the trade-off between embedding dimension and retrieval quality?
- What are 'contextual embeddings' and why are they useful?
- How does Retrieval-Augmented Generation (RAG) work?
- What are text embeddings and how are they used?
- How should you chunk documents for RAG?
- Fine-tuning vs RAG — which do you use when?