Exploding Gradients
Gradients growing exponentially large as they propagate backward through many layers, producing huge parameter updates that destabilize or crash training.
The mirror image of vanishing gradients: when the per-layer multiplier in the backward pass is consistently greater than one, the product grows exponentially with depth instead of shrinking. A weight update that should nudge a parameter can instead throw it somewhere the loss has never sampled, and the very next forward pass returns nan.
Recurrent networks unrolled over long sequences hit this hardest, since the same weight matrix multiplies in at every step. Gradient clipping is the direct fix; careful initialization and normalization layers reduce how often it's needed in the first place.