What inductive biases do CNNs have?
mediumAnswer
- (1) Locality: convolutions look at small neighborhoods.
- (2) Translation equivariance: shifting the input shifts the feature map.
- (3) Weight sharing: same filter applied everywhere, so few parameters.
- (4) Hierarchy via stacking: early layers learn edges, later layers learn parts and objects.
- These biases make CNNs data-efficient for images compared to plain MLPs.
Check yourself — multiple choice
- CNNs assume global connectivity between all pixels
- CNNs have locality, weight sharing, and translation equivariance
- CNNs are rotation-invariant by default
- CNNs have no inductive biases
Locality + weight sharing + translation equivariance = CNN inductive biases.
#cnn#computer-vision
Practise Deep Learning
214 interview questions in this topic.
Related questions
- What is ConvNeXt's philosophy?
- What is the receptive field in a CNN?
- Compute the output shape of a 2D convolution with input (H, W), kernel k, stride s, padding p, dilation d.
- SAME vs VALID vs REFLECT padding — what are the practical differences?
- What is a dilated (atrous) convolution?
- What is a transposed convolution ('deconv') and its checkerboard artifact issue?