Zero-shot image classification via CLIP — mechanism.
mediumAnswer
- (1) Encode candidate class names as text ('a photo of a dog', 'a photo of a cat') → text embeddings.
- (2) Encode query image → image embedding.
- (3) Cosine similarity → class with max similarity is prediction.
- No training on class labels.
- Weakness: sensitive to prompt engineering; competitive with fine-tuned models on many benchmarks.
- Foundation of open-vocabulary classification and detection (OWL-ViT, GLIP).
Check yourself — multiple choice
- Same as fine-tune
- Encode class names as 'a photo of {class}' text + image; cosine max = prediction; no label training; prompt-sensitive; foundation of open-vocab detection (OWL-ViT, GLIP)
- Random
- Not real
CLIP zero-shot: text prompts + image + cosine max; foundation of open-vocab tasks.
#applications
Practise Unsupervised Learning
214 interview questions in this topic.