Does position in the context matter for what the LLM 'sees'?
mediumAnswer
- Yes — Liu et al. (2023) 'Lost in the Middle' showed LLMs use context near the start and end far more effectively than in the middle.
- On a QA task with 20 retrieved docs, accuracy drops 20+% when the answer is in doc 10 vs doc 1 or 20.
- Practical implications: (1) put the most important context near the beginning or end; (2) shorter, more relevant context beats stuffing everything in; (3) re-rank retrieved docs and put the top few near the query.
Check yourself — multiple choice
- Position doesn't matter
- 'Lost in the middle' — LLMs attend better to start/end; put important info at edges; re-rank retrieved docs
- Only end matters
- Random position best
Lost in the middle: put critical context at start / end; re-rank for RAG.
#prompting#long-context#rag
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- How do you counter 'lost in the middle' in RAG?
- Why does stuffing more context into a long-context model sometimes make answers worse?
- How does Retrieval-Augmented Generation (RAG) work?
- How should you chunk documents for RAG?
- What are the pillars of good prompt engineering?
- Fine-tuning vs RAG — which do you use when?