What is a computation graph and why does autograd care?
easyAnswer
- A DAG whose nodes are operations and edges track data + gradient dependencies.
- During the forward pass, each op records its inputs so its backward function can be called.
- Reverse-mode autodiff (backprop) traverses the graph from the loss backwards, applying the chain rule at each node.
- PyTorch builds it dynamically per iteration; TensorFlow (TF1) and JAX build it statically.
Check yourself — multiple choice
- A graph of neural-net layers only
- A DAG of ops used by autograd to compute gradients via reverse-mode chain rule
- Only used at inference
- The same as a tree structure
Computation graph = DAG that autograd traverses to compute gradients.
#backprop#autograd#fundamentals
Practise Deep Learning
214 interview questions in this topic.
Related questions
- In one sentence, what is backpropagation?
- ReLU vs sigmoid vs GELU — when do you use each?
- State the universal approximation theorem in one sentence and its practical caveat.
- Why does stacking linear layers without nonlinearity collapse to a single linear layer?
- Depth vs width — which do you scale first?
- Why can't you initialize a neural net with all zeros?