What is Matryoshka Representation Learning (MRL)?
hardAnswer
- Train the embedding model with a loss that operates at multiple truncation dims simultaneously (256, 512, 1024, 2048) — the first k dimensions must independently be a good embedding for any k.
- At inference, you can slice to any size without retraining.
- Enables cheap-quality trade-offs: index at 1024 for search, truncate to 256 for a 4x smaller cache or a faster reranker input.
- Adopted by OpenAI, Cohere, Nomic, Jina.
Check yourself — multiple choice
- Fixed dimension
- Train so any prefix of dims is a valid embedding → slice to any size at inference (OpenAI v3, Nomic, Jina)
- Only for images
- Impossible to shorten
MRL: nested embeddings — any prefix works → flexible dim at inference.
#embeddings
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- What are text embeddings and how are they used?
- How do you pick an embedding model for a new RAG project?
- What's the trade-off between embedding dimension and retrieval quality?
- How is ColBERT different from a standard bi-encoder?
- What are 'contextual embeddings' and why are they useful?
- How does multimodal retrieval (image + text) work?