What is Xavier (Glorot) initialization and why?
mediumAnswer
- Sample weights from a distribution with variance — designed so that activations and gradients have roughly the same variance layer to layer for tanh/sigmoid activations.
- Prevents both vanishing and exploding signals at initialization.
- Default in older frameworks; replaced by He init when ReLU became the standard.
Check yourself — multiple choice
- Xavier init works best for ReLU networks
- Xavier uses to keep signal variance stable across tanh layers
- Xavier initializes all weights to zero
- Xavier depends on batch size
Xavier: variance — for tanh/sigmoid.
#initialization#training
Practise Deep Learning
214 interview questions in this topic.
Related questions
- What are vanishing and exploding gradients, and how do you fix them?
- What does Batch Normalization do?
- How does dropout work and when is it applied?
- Why use a learning-rate schedule (warmup + cosine decay)?
- Why do residual (skip) connections help training deep networks?
- Why does He initialization use Var = 2 / fan_in instead of Xavier's 2 / (fan_in + fan_out)?