EasyDeepLearn
Deep Learning · section 10 of 19

Convolutional networks

11 interview questions on convolutional networks, each answered in full. Free to read, no account needed.

What inductive biases do CNNs have?

medium
  • (1) Locality: convolutions look at small neighborhoods.
  • (2) Translation equivariance: shifting the input shifts the feature map.
  • (3) Weight sharing: same filter applied everywhere, so few parameters.
  • (4) Hierarchy via stacking: early layers learn edges, later layers learn parts and objects.
  • These biases make CNNs data-efficient for images compared to plain MLPs.
#cnn#computer-visionPermalink & quiz →

What is the receptive field in a CNN?

medium
  • The receptive field of a neuron is the region of the input image that influences its value.
  • It grows with depth, kernel size, and stride/dilation.
  • Deeper layers see larger regions.
  • Techniques to grow the receptive field: stacking layers, larger kernels, strided convolutions, dilated (atrous) convolutions, and pooling.
  • A large enough receptive field is required to capture global context.
#cnnPermalink & quiz →

SAME vs VALID vs REFLECT padding — what are the practical differences?

easy
  • VALID = no padding (output shrinks each layer).
  • SAME = pad zeros so output has same H, W as input for stride 1.
  • REFLECT = pad by reflecting pixels near the boundary (avoids the 'black-edge' artifacts you get with zero-padding in segmentation / super-resolution).
  • REPLICATE = pad by repeating the edge pixel.
  • Choice matters for boundary quality: SAME/zero is fine for classification, REFLECT/REPLICATE for dense prediction.
#cnnPermalink & quiz →

What is a dilated (atrous) convolution?

medium
  • Insert (d-1) zeros between kernel elements to enlarge the receptive field without adding parameters or reducing resolution.
  • A 3×3 kernel with dilation 2 has an effective 5×5 view.
  • Standard trick in segmentation networks (DeepLab, HRNet) where downsampling would lose spatial detail.
  • Stacked dilated convs with growing dilation cover a large receptive field cheaply — also used in WaveNet for 1D audio.
#cnn#segmentationPermalink & quiz →

What is a transposed convolution ('deconv') and its checkerboard artifact issue?

hard
  • A convolution that upsamples by inserting zeros between pixels then applying a normal conv — output is larger than input.
  • Used in decoders (U-Net upsampling), GAN generators, and dense prediction.
  • Checkerboard artifacts occur when kernel size isn't divisible by stride: some output pixels get contributions from more kernel positions than others.
  • Fix: use bilinear upsample + normal 3x3 conv, or ensure kernelsize  =  2  ⋅  stride\mathrm{kernel}_{\mathrm{size}}\; = \;2\; \cdot \;\mathrm{stride}.
#cnn#generative#segmentationPermalink & quiz →

How does a depthwise separable convolution reduce compute?

medium
  • Factor a standard conv into (1) depthwise: one k×k filter per input channel independently; (2) pointwise: a 1×1 conv mixing the depthwise outputs across channels.
  • Compute goes from Cin⋅Cout⋅k⋅kC_{\mathrm{in}} \cdot C_{\mathrm{out}} \cdot k \cdot k to Cin⋅k⋅k  +  Cin⋅CoutC_{\mathrm{in}} \cdot k \cdot k\; + \;C_{\mathrm{in}} \cdot C_{\mathrm{out}} — for k=3, Cin=Cout=512C_{\mathrm{in}} = C_{\mathrm{out}} = 512, that's 8-9x cheaper.
  • Foundation of MobileNet, Xception, EfficientNet.
  • Small accuracy loss vs standard convs but huge speedups on mobile / embedded.
#cnn#architecturesPermalink & quiz →

What are 1x1 convolutions used for?

easy
  • Three roles: (1) channel-mixing / channel-wise projection — a matrix multiply across channels per spatial location, so it's a cheap way to change the number of channels; (2) bottleneck in ResNet blocks (reduce channels → 3×3 conv → expand channels) to save compute; (3) pointwise conv in depthwise-separable architectures.
  • Effectively a per-position linear layer.
#cnnPermalink & quiz →

Max pooling vs average pooling — when do you pick each?

easy
  • Max pool: keeps the strongest activation in the window — sharper features, invariance to small translations, dominant in classification CNNs.
  • Average pool: smoother, retains overall energy — better in generative / decoder paths where you don't want to lose detail.
  • Modern architectures often replace pool with strided convs (learnable downsampling).
  • Global average pool (GAP) at the end of a CNN replaces the fully-connected head and reduces overfitting.
#cnnPermalink & quiz →

Why does Global Average Pooling replace the FC head in modern CNNs?

medium
  • Take the mean of each feature map across H, W to get a C-dim vector, then apply a linear classifier.
  • Advantages: (1) drastic parameter reduction — no huge FC on flattened features; (2) intrinsic regularization; (3) enables class activation maps (CAM) since one channel per class is aligned with a class score; (4) works with any input resolution.
  • Introduced in Network-in-Network and popularized by ResNet.
#cnn#architecturesPermalink & quiz →

What is adaptive pooling and why is it useful?

easy
  • Given a target output size (Hout,  Wout)(H_{\mathrm{out}}, \;W_{\mathrm{out}}), computes the kernel and stride needed to produce that output regardless of input size.
  • Standard trick to accept variable input resolutions with a fixed-size classifier head. torch.nn.AdaptiveAvgPool2d((1,1)) is the go-to for global pooling.
  • Also useful for detection networks that need fixed-size feature maps for RoI heads.
#cnnPermalink & quiz →

What is ConvNeXt's philosophy?

hard
  • Liu et al. (2022) 'modernize' a ResNet using tricks from Swin: bigger kernels (7×7 depthwise), GELU + LayerNorm, inverted bottlenecks, fewer activations / norms, and modern training recipes (AdamW + Mixup + RandAugment).
  • The result is a pure-conv architecture that matches Swin Transformer on ImageNet, detection, and segmentation.
  • Shows attention isn't magical — modern training + design does most of the work.
#architectures#computer-vision#cnnPermalink & quiz →

Practise Deep Learning

All 19 sectionsFlashcardsQuiz