Backpropagation
The foundational algorithm used to train neural networks by calculating the gradient of the loss function with respect to every single weight in the network.
Think of It Like This
Like tracing a flawed product on an assembly line backward through every single factory station to find exactly which worker made the mistake.
It works by applying the chain rule of calculus backward from the output layer to the input layer. By knowing exactly how much each parameter contributed to the final error, optimization algorithms like Adam can adjust those weights to improve future predictions. Without backpropagation, training deep architectures would be mathematically intractable.