Word embeddings as unsupervised DR of text.
mediumAnswer
- Word2Vec (skip-gram, CBOW), GloVe: predict context words → learn dense word vectors capturing co-occurrence structure.
- Word2Vec is implicit PMI matrix factorization (Levy & Goldberg 2014).
- GloVe: explicit log-count factorization.
- FastText: adds sub-word info for OOV.
- Modern replacement: contextual embeddings (BERT, sentence-transformers) — the dominant text representation since 2018.
Check yourself — multiple choice
- Random
- Word2Vec / GloVe / FastText factorize co-occurrence matrix into dense word vectors; contextual embeddings (BERT) dominate since 2018
- Same as t-SNE
- Not DR
Word embeddings: dense DR of co-occurrence matrix; contextual embeddings modern.
#dimensionality-reduction#nlp#representation-learning
Practise Unsupervised Learning
214 interview questions in this topic.