How does a Vision Transformer (ViT) treat images?
hardAnswer
- Split image into fixed-size patches (e.g., 16×16), linearly embed each patch (like word embeddings), add positional embeddings, prepend a learnable [CLS] token, and feed to a standard transformer encoder.
- The CLS token's final representation is the image feature for classification.
- Requires huge pretraining data (JFT-300M) to match CNN accuracy — with less data, CNNs still win.
- DeiT (Touvron 2021) added distillation + augmentation to make ViT competitive at ImageNet-scale.
Check yourself — multiple choice
- ViT uses convolutions
- Split image into patches, linearly embed + position + [CLS], feed to a transformer — needs big data or distillation
- ViT has no positional info
- ViT is deprecated
ViT: patch embedding + positional + [CLS] token → standard transformer.
#transformers#computer-vision#architectures
Practise Deep Learning
214 interview questions in this topic.