What does Swin Transformer do differently from ViT?
hardAnswer
- Swin uses hierarchical (multi-scale) feature maps like CNNs, with attention computed inside local windows (e.g., 7×7 patches).
- Adjacent transformer blocks shift the windows so information flows across window boundaries — hence 'Swin' (Shifted Windows).
- O(N) complexity in image size vs ViT's , and produces multi-scale features suitable for detection / segmentation.
Check yourself — multiple choice
- Global attention like ViT
- Local windowed attention + shifted windows across layers → hierarchical, O(N) in image size
- No positional info
- Only for text
Swin: shifted windows + hierarchical maps → linear complexity, dense-prediction friendly.
#transformers#computer-vision#architectures
Practise Deep Learning
214 interview questions in this topic.