Dropout
A regularization technique where randomly selected neurons are temporarily disabled during training to prevent the network from relying on specific connections.
Think of It Like This
Like a sports team practicing with different starting players every day so they don't become overly dependent on just one superstar to win the game.
By dropping out units with a certain probability, the neural network is forced to learn redundant and robust representations. This prevents complex co-adaptations where neurons fix the mistakes of their neighbors. During inference, all neurons are active, but their weights are scaled down to account for the training behavior.