Weight Initialization
The scheme for picking a network's starting weights, chosen so activations and gradients keep a stable scale across layers instead of collapsing or exploding.
Every weight starts at a small random number, never zero, because identical weights would compute identical gradients forever and no unit would ever specialize. How large that random number should be is the actual question, and it depends on how many inputs feed the unit: too small and every layer's output shrinks toward zero, too large and it blows up.
Xavier initialization balances the variance for a linear or tanh unit; He initialization doubles that variance to account for ReLU zeroing out half the values. Both scale down as the layer gets wider, which is the part people forget when they hand-pick a single fixed standard deviation for every layer in a network.