How is RAG retrieval an unsupervised problem?
mediumAnswer
- RAG (retrieval-augmented generation) retrieves passages from a corpus based on unsupervised embeddings (no labels of 'relevant' pairs).
- Uses: dense retrieval (sentence embeddings + cosine + ANN), often hybrid with BM25.
- Fine-tuning the retriever is supervised (via query-doc labels), but out-of-the-box embeddings + cosine already give strong retrieval unsupervised.
- Foundation of modern LLM-augmented systems.
Check yourself — multiple choice
- Fully supervised
- Uses unsupervised embeddings (sentence-transformers, E5, OpenAI-3) + cosine + ANN; hybrid with BM25; strong retrieval out-of-the-box before any fine-tune
- Random
- Same as GPT
RAG retrieval: unsupervised embeddings + cosine + ANN; SFT is extra.
#applications#nlp
Practise Unsupervised Learning
214 interview questions in this topic.
Related questions
- BM25 — why is it still competitive with modern retrievers?
- Vector search / approximate nearest neighbors — main algorithms.
- Applications of clustering in NLP.
- How do you cluster / retrieve code snippets?
- Interview: users complain search returns irrelevant results — how do you fix?
- Interview: choose a sentence embedding model for a startup RAG.