EasyDeepLearn
Lesson

Deep Learning

214 questions19 sections

Neural networks, training tricks, CNNs, RNNs, transformers and everything in between.

Every questionFlashcardsTake the quiz

Introduction

Deep learning is what happens when you stack differentiable layers deep enough — and add enough data, compute, and clever tricks — for gradient descent to discover useful representations by itself.

Interview questions here fall into three buckets: (1) the numerical and optimization tricks that make training work (initialization, normalization, activations, learning-rate schedules); (2) architectures with useful inductive biases (CNNs for images, transformers for sequences); (3) understanding *why* certain choices dominate the modern stack. This chapter walks through the essentials without hand-waving.

The 19 sections

Each section is a short read on one subject, with every answer written out. Work through them in order, or jump to the one you are weakest on.

  1. 01Activations & fundamentalsReLU vs sigmoid vs GELU — when do you use each?25 questions12 easy12 medium1 hard
  2. 02Backpropagation & autogradWhy does the backward pass require more memory than the forward pass?2 questions1 easy1 medium
  3. 03Initialization & gradientsWhat are vanishing and exploding gradients, and how do you fix them?6 questions1 easy4 medium1 hard
  4. 04Training dynamicsWhat does Batch Normalization do?43 questions6 easy23 medium14 hard
  5. 05OptimizersSGD vs Adam vs AdamW — how do you choose?17 questions2 easy8 medium7 hard
  6. 06Learning-rate schedulesWhat are cyclical learning rates (CLR)?7 questions2 easy4 medium1 hard
  7. 07NormalizationWhen do you use LayerNorm instead of BatchNorm?9 questions7 medium2 hard
  8. 08RegularizationWhat is DropConnect and how is it different from Dropout?9 questions3 easy3 medium3 hard
  9. 09Data augmentation & mixingHow does CutMix differ from Mixup?3 questions1 easy2 medium
  10. 10Convolutional networksWhat inductive biases do CNNs have?11 questions4 easy5 medium2 hard
  11. 11CNN architecturesWhat made AlexNet (2012) a breakthrough?27 questions2 easy11 medium14 hard
  12. 12Object detection & segmentationWhat are IoU / Jaccard and Tversky losses?1 questions1 hard
  13. 13Recurrent networksWhy did transformers replace RNNs for sequence modeling?8 questions1 easy5 medium2 hard
  14. 14Attention & transformersHow does self-attention work in a transformer?22 questions7 medium15 hard
  15. 15Losses for deep learningWrite the Huber loss and explain when to use it.11 questions2 easy4 medium5 hard
  16. 16Transfer learning & fine-tuningWhat is transfer learning and when is fine-tuning better than feature extraction?3 questions2 medium1 hard
  17. 17Efficiency & distributed trainingWhat is gradient checkpointing and its trade-off?2 questions1 medium1 hard
  18. 18Generative modelsWhat is mode collapse in GANs and how do you mitigate it?4 questions4 hard
  19. 19Compression & interpretabilityHow does knowledge distillation work?4 questions3 medium1 hard