Definition

Identity Matrix

A square matrix with ones down the diagonal and zeros elsewhere, which leaves any matrix it multiplies unchanged.

Think of It Like This

Multiplying by one, for matrices.

Written $I$ , the identity matrix satisfies $AI = IA = A$ for any compatible $A$ . It is the multiplicative identity of matrix algebra, the same role the number 1 plays for ordinary multiplication.

Its practical uses are mostly structural. It defines the matrix inverse — $A^{-1}$ is whatever matrix gives $AA^{-1} = I$ . It is what a residual connection adds, which is why residual blocks start out close to passing their input through untouched. And adding a small multiple of it, $A + \epsilon I$ , is the standard trick for making a near-singular matrix invertible, which is exactly what ridge regression does to the normal equations.