Word embedding analogies — why do they work?
hardAnswer
- king - man + woman ≈ queen.
- Emerges because embeddings capture distributional differences.
- Actually more fragile than the meme suggests: only works for common frequent analogies, and typical evaluations use cosine similarity + exclusion of query words.
- Modern contextual embeddings capture analogy implicitly through generation.
- Interview trap: don't overstate this as evidence of 'reasoning'.
Check yourself — multiple choice
- Random
- Emerges from distributional differences; works on common analogies with cosine + exclusion of query words; less general than the meme — not 'reasoning'
- Same as GPT reasoning
- Not real
Word analogies: distributional emergence; less general than popular meme suggests.
#representation-learning#nlp
Practise Unsupervised Learning
214 interview questions in this topic.