Residual Connection
An architectural shortcut in deep neural networks that entirely bypasses one or more layers by directly adding the original input to the layer's output.
Think of It Like This
Like a highway bypass that lets cars skip the congested city center and merge directly back onto the main road on the other side.
By providing an unobstructed gradient path back to earlier layers, residual connections solve the degradation problem where ultra-deep networks perform worse than shallow ones. Mathematically represented as F(x) + x, they are the defining innovation of ResNet and are an absolutely mandatory component in every modern transformer architecture.