In one sentence, what is backpropagation?
easyAnswer
- Backpropagation is the reverse-mode automatic differentiation algorithm that efficiently computes gradients of a scalar loss with respect to every parameter of a neural network by traversing the computation graph backwards, applying the chain rule at each node.
Check yourself — multiple choice
- Backprop computes weights directly without gradients
- Backprop applies the chain rule in reverse through the compute graph
- Backprop is a specific optimizer like Adam
- Backprop only works for CNNs
Backprop = reverse-mode autodiff = chain rule applied in reverse.
#backprop#fundamentals
Practise Deep Learning
214 interview questions in this topic.
Related questions
- What is a computation graph and why does autograd care?
- ReLU vs sigmoid vs GELU — when do you use each?
- State the universal approximation theorem in one sentence and its practical caveat.
- Why does stacking linear layers without nonlinearity collapse to a single linear layer?
- Depth vs width — which do you scale first?
- Why can't you initialize a neural net with all zeros?