Should you fine-tune the LLM for RAG or improve retrieval first?
mediumAnswer
- Almost always improve retrieval first — it's cheaper, more auditable, and 80% of RAG problems are retrieval problems (wrong chunks, missing chunks, poor recall).
- Fine-tune when: (1) the LLM refuses / hallucinates despite good retrieval; (2) the domain has heavy jargon the base LLM stumbles on; (3) you need a specific citation / format style.
- Debug order: retrieval eval → prompt tuning → reranking → fine-tune generator only if the previous fail.
Check yourself — multiple choice
- Always fine-tune first
- Improve retrieval first (80% of RAG issues are retrieval); fine-tune LLM only if retrieval + prompt + rerank aren't enough
- Only fine-tune
- Random order
Debug order: retrieval → prompt → rerank → LLM fine-tune.
#rag#fine-tuning
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- Fine-tuning vs RAG — which do you use when?
- A stakeholder wants the model to 'know our internal docs'. Do you fine-tune or build RAG?
- How does Retrieval-Augmented Generation (RAG) work?
- How should you chunk documents for RAG?
- What is LoRA and why is it popular for fine-tuning LLMs?
- What is continual pretraining and when does it beat fine-tuning?