How does DETR reformulate object detection?
hardAnswer
- DETR (Carion 2020) treats detection as a set-prediction problem: a transformer decoder outputs a fixed set of N (>>true count) predictions in parallel, matched to ground-truth objects via bipartite Hungarian matching during training.
- No anchors, no NMS.
- Simple, elegant, but slow to converge (500+ epochs) and struggles on small objects.
- Deformable-DETR fixes both by using deformable attention and multi-scale features.
How to say it out loud
DETR treats detection as set prediction. The decoder emits a fixed number of predictions in parallel — a hundred, say — and during training a Hungarian matcher assigns each one either to a ground-truth box or to the 'no object' class. That one-to-one matching is what removes anchors and NMS: the model cannot produce duplicates, so there is nothing to suppress afterwards. The price is convergence. The original needed something like five hundred epochs, and small objects suffered because the encoder works at a single low-resolution feature map. Deformable DETR fixed both by attending to a few sampled points across multiple scales.
Check yourself — multiple choice
- DETR uses anchors and NMS
- Transformer decoder outputs a fixed set of predictions matched to GT via Hungarian — no anchors, no NMS
- Same as Faster R-CNN
- Slower than Faster R-CNN inference
DETR: set prediction via transformer + Hungarian matching → no anchors, no NMS.
What the interviewer asks next
- What is Non-Maximum Suppression (NMS) and its variants?The natural next question is what exactly DETR made unnecessary, and whether you can describe it.
- How does a two-stage detector (Faster R-CNN) work?The claim only means something against the anchor-based pipeline it replaced.
- How does YOLO / SSD / RetinaNet differ from two-stage detection?Places DETR on the map: it is not a one-stage detector either, and being able to say why is the point.
- Why is standard self-attention O(n²) in sequence length?The single-scale feature map and the slow convergence both trace back to what attention costs.
Practise Deep Learning
214 interview questions in this topic.