How do you choose a chunking strategy for RAG?
mediumAnswer
- Chunk on the document's own structure before falling back to size.
- Headings, sections and list items produce chunks that are semantically whole, which matters more than hitting an exact token count.
- Aim for a size that comfortably fits several chunks in the context budget, commonly a few hundred tokens, with a modest overlap so a sentence split across a boundary is still recoverable.
- Keep the parent heading in each chunk's text so an isolated paragraph is still interpretable.
- Then evaluate: retrieval recall on a labelled question set is the only way to know whether your chunking works, and it usually beats intuition.
Check yourself — multiple choice
- Always fixed 512 tokens
- Split on document structure first, add overlap and parent headings, then validate with retrieval recall on labelled questions
- One chunk per document
- Chunking does not matter
Structure-aware chunks with context headers, validated by measured retrieval recall.
#rag#retrieval
Practise LLMs & GenAI
214 interview questions in this topic.