A stakeholder wants the model to 'know our internal docs'. Do you fine-tune or build RAG?
mediumAnswer
- RAG, in almost every case.
- Fine-tuning teaches style, format and task behaviour; it is a poor way to install facts, because the knowledge is baked in at training time and cannot be updated, cited or revoked.
- RAG keeps documents in a store you can re-index nightly, lets you show sources so the answer is auditable, and handles access control because you can filter retrieval by permission.
- Fine-tuning becomes the right answer when you need a specific output schema, a domain tone, or lower latency from a smaller model.
- The two combine well: fine-tune for behaviour, retrieve for facts.
Check yourself — multiple choice
- Always fine-tune
- RAG for facts — updatable, citable, permission-filterable; fine-tune for format, tone and latency, and combine them
- Neither works
- Fine-tuning is cheaper for facts
Facts belong in retrieval; fine-tuning shapes behaviour and format.
#rag#fine-tuning
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- Fine-tuning vs RAG — which do you use when?
- Should you fine-tune the LLM for RAG or improve retrieval first?
- How does Retrieval-Augmented Generation (RAG) work?
- How should you chunk documents for RAG?
- What is LoRA and why is it popular for fine-tuning LLMs?
- What is continual pretraining and when does it beat fine-tuning?