What is a cross-encoder reranker and when do you need one?
mediumAnswer
- Take top-N (~50-200) candidates from the retriever, feed each (query, doc) pair through a cross-encoder (encode query and doc jointly, output relevance score).
- Much higher precision than bi-encoder retrieval because attention can compare tokens directly.
- But slow — impossible at index scale.
- Standard: retrieve 100, rerank to top-5 or top-10.
- Popular rerankers: Cohere Rerank, BGE-Reranker, MixedBread mxbai-rerank, ColBERT-v2.
- Adds 100-500ms latency but 10-20% precision@k gain.
Check yourself — multiple choice
- Skip reranking always
- Cross-encoder scores (query, doc) jointly on top-N candidates → 10-20% precision@k gain at ~100ms latency
- Same as bi-encoder
- Only for training
Reranker: cross-encoder over top-N → big precision gain at moderate latency cost.
#retrieval#reranking
Practise LLMs & GenAI
214 interview questions in this topic.