Why re-rank retrieved documents?
mediumAnswer
- Initial retrieval (bi-encoder embedding) is fast but coarse.
- Re-ranker (cross-encoder, considers query + doc together) is much more accurate but slow — run only on top-K candidates.
- Boost NDCG by 10-30%.
- Popular: Cohere Rerank API, BGE-reranker (OSS), Voyage Rerank, mixedbread rerank.
- Trade-off: latency + cost vs quality.
- Modern stack: hybrid retrieve top-100 → rerank top-20 → LLM sees top-5.
Check yourself — multiple choice
- Random
- Bi-encoder retrieval fast but coarse; cross-encoder reranker considers query+doc together, more accurate but slow → run on top-K; boost NDCG 10-30%; Cohere/BGE/Voyage/mixedbread; hybrid→rerank→top-5 for LLM
- Not useful
- Not real
Rerank: cross-encoder on top-K; NDCG +10-30%; Cohere / BGE / Voyage.
#llmops
Practise MLOps & Data Quality
215 interview questions in this topic.