What are 'hard negatives' in embedding training?
hardAnswer
- Negatives that look superficially similar to the positive but are actually irrelevant — force the model to learn fine-grained distinctions.
- Mining strategies: (1) BM25 top-k excluding the positive (fast, common); (2) previous-generation retriever's top-k excluding the positive (harder); (3) LLM-generated distractors ('write a passage that's topically similar but doesn't answer the query').
- Better than random in-batch negatives, which are almost always easy.
- Standard in modern embedding training (E5, BGE, GTE).
Check yourself — multiple choice
- Random negatives suffice
- Superficially similar but irrelevant → mine via BM25 top-k / prior retriever / LLM distractors; standard in modern embedding training
- Only for images
- Same as positives
Hard negatives: superficially similar but wrong → sharper decision boundaries.
#embeddings#fine-tuning
Practise LLMs & GenAI
214 interview questions in this topic.
Related questions
- When and how do you fine-tune an embedding model?
- What are text embeddings and how are they used?
- Fine-tuning vs RAG — which do you use when?
- What is LoRA and why is it popular for fine-tuning LLMs?
- What is continual pretraining and when does it beat fine-tuning?
- What is supervised fine-tuning (SFT) and where does it fit in the alignment pipeline?