How should you chunk documents for RAG?
mediumAnswer
- Split documents into semantically coherent chunks around 300-800 tokens (varies by embedding model) with 10-20% overlap so context is not cut mid-idea.
- Preserve structural boundaries (headings, sections) when possible; add metadata like source and page.
- Too small chunks lose context; too large chunks dilute relevance and inflate cost.
- Iterate: evaluate retrieval recall on real queries and tune.
Check yourself — multiple choice
- Always use single-sentence chunks
- Around 300-800 tokens with some overlap works as a starting point
- Chunk size does not affect retrieval quality
- Chunks must never overlap
A few hundred tokens with a small overlap is the pragmatic default.
#rag#retrieval
Practise LLMs & GenAI
214 interview questions in this topic.