How do modern sentence / doc embeddings work?
mediumAnswer
- Sentence-BERT (Reimers-Gurevych): fine-tune BERT with siamese contrastive loss on paraphrase / NLI pairs → semantically-meaningful vectors, cosine similarity ≈ semantic similarity.
- Modern: E5, BGE, GTE, OpenAI text-embedding-3, Cohere embed-v3 — trained on contrastive tasks over huge multilingual corpora.
- Used everywhere in RAG, retrieval, semantic search, clustering.
Check yourself — multiple choice
- Random
- Sentence-BERT / E5 / BGE / OpenAI: contrastive fine-tuning of encoders on NLI/paraphrase pairs → cosine ≈ semantic similarity; core of RAG / retrieval
- Same as PCA
- Not real
Modern sentence embeddings: contrastive fine-tuned encoders; RAG / retrieval.
#dimensionality-reduction#nlp#representation-learning
Practise Unsupervised Learning
214 interview questions in this topic.