Sentence-BERT — how does it improve on BERT for retrieval?
mediumAnswer
- BERT's [CLS] token is not a great semantic sentence vector out of the box.
- SBERT fine-tunes BERT with a siamese architecture on NLI + STS: same encoder on two sentences, minimize distance for paraphrases, maximize for contradictions.
- Cosine similarity ≈ semantic similarity.
- Enables efficient sentence retrieval (encode once, cosine at query time), replaced by newer E5 / BGE / OpenAI text-embedding-3 for scale, but SBERT is still baseline.
Check yourself — multiple choice
- Same as BERT
- Siamese BERT fine-tuned on NLI + STS → cosine ≈ semantic similarity; enables efficient retrieval; superseded by E5 / BGE / OpenAI-3 at scale
- Random
- Not real
SBERT: siamese-BERT fine-tuned for semantic similarity; retrieval foundation.
#representation-learning#nlp
Practise Unsupervised Learning
214 interview questions in this topic.