What's the trade-off between embedding dimension and retrieval quality?
mediumAnswer
- Higher dim = more expressive space, marginally better recall on hard queries, more storage, slower ANN search.
- E.g., OpenAI text-embedding-3-large supports 3072 dim but can be shortened to 256/512/1024 via Matryoshka Representation Learning (MRL) — a training trick that keeps quality high at lower dims.
- Practical: 768-1024 is the sweet spot for most RAG; go higher only if benchmarks justify it.
- Storage matters at scale: 3072-dim × 4 bytes × 10M vectors = 120 GB.
Check yourself — multiple choice
- Higher dim always better
- Higher dim: marginal recall gain, more storage / slower ANN; 768-1024 sweet spot; MRL enables shortening without quality loss
- Dimension doesn't matter
- Lower dim always wins
Embedding dim: 768-1024 sweet spot; higher = more storage / slower ANN; MRL enables shortening.
#embeddings#retrieval#vector-db
Practise LLMs & GenAI
214 interview questions in this topic.