What is Neural Architecture Search (NAS)?
hardAnswer
- Automated search for the best architecture (layers, connections, widths) given a target task and constraints (FLOPs, latency, memory).
- Methods: reinforcement learning over architectures (NASNet, MnasNet), evolutionary search (AmoebaNet), gradient-based (DARTS — differentiable), or one-shot supernets (Once-For-All).
- Produced EfficientNet, MobileNetV3, MnasNet — SOTA vision architectures under mobile budgets.
- Now less used for LLMs (transformers are already close to optimal at scale); still active in edge vision and hardware-aware design.
Check yourself — multiple choice
- Manual architecture design
- Automated architecture search (RL / evolution / gradient / supernet) → EfficientNet, MobileNetV3, MnasNet
- NAS is deprecated
- Only for CNNs at ImageNet scale
NAS: automated architecture discovery under FLOPs/latency budgets.
#architectures
Practise Deep Learning
214 interview questions in this topic.
Related questions
- Why do residual (skip) connections help training deep networks?
- Depth vs width — which do you scale first?
- How do you compute the parameter count of a linear layer and a convolution?
- How do FLOPs scale for a forward pass through an MLP and a transformer?
- What is stochastic depth / DropPath and where is it used?
- How does a depthwise separable convolution reduce compute?