What is self-supervised learning and why does it matter?
mediumAnswer
- Self-supervised learning creates labels from the data itself via pretext tasks (masked language modeling, next-token prediction, contrastive views of images) — no manual annotation.
- It matters because it lets us pretrain huge models on unlabeled data and then transfer to downstream tasks with little labeled data.
- It's the engine behind BERT, GPT, SimCLR, CLIP, MAE, DINO.
Check yourself — multiple choice
- It requires manual labels
- It generates labels from the data itself (pretext tasks)
- It is the same as supervised learning
- It only works on images
Self-supervision invents labels from the data (mask, contrast, predict-next).
#representation-learning
Practise Unsupervised Learning
214 interview questions in this topic.