Symmetric Eigendecomposition

Why Does This Exist?

Covariance matrices, Hessians, kernels and graph Laplacians share one property: symmetry ($A^T = A$). Symmetry buys the strongest decomposition in linear algebra. Where a general matrix needs a fragile $PDP^{-1}$ with an explicit inverse, a symmetric matrix splits into $A = Q\Lambda Q^T$ with $Q$ orthogonal, so the inverse is a free transpose. Eigenvalues come out real, eigenvectors out mutually perpendicular, and nothing can go complex or defective on you.

That is why PCA works on the covariance without fear, why the Hessian curvature directions are trustworthy, and why spectral clustering leans on the Laplacian. General eigenvalues and eigenvectors teach the objects; this page is the symmetric upgrade where everything becomes orthonormal.

The one idea you need first

An orthogonal matrix $Q$ satisfies $Q^T Q = I$: its columns are unit length and mutually perpendicular, and $Q^{-1} = Q^T$. Rotations and reflections qualify. Multiplying by $Q$ never stretches anything, it only re-views, which is why the symmetric decomposition is numerically bulletproof.

Think of It Like This

How It Actually Works

For symmetric $A$, there exist orthonormal $q_1, \ldots, q_n$ and real $\lambda_i$ with:

In plain words: $Q$ holds the axes as columns, $\Lambda$ holds the stretches on its diagonal. Read it right to left on a vector $x$: $Q^T$ re-views $x$ in the eigenbasis, $\Lambda$ stretches each coordinate by $\lambda_i$, $Q$ rotates back. The sum form says $A$ is a weighted stack of rank-one projections, weight $\lambda_i$ on direction $q_i$. Dropping small-$\lambda_i$ terms is exactly truncated PCA, and the spectral theorem states why this always works.

Worked example

Split $A = \begin{bmatrix} 4 & 1 \\ 1 & 3 \end{bmatrix}$. Hand-computed trace is $4 + 3 = 7$ and determinant is $4 \times 3 - 1 \times 1 = 12 - 1 = 11$, so the characteristic equation $\lambda^2 - 7\lambda + 11 = 0$ gives:

Both real, as symmetry guarantees. Their product $4.618 \times 2.382 \approx 11.0$ recovers the determinant, and their sum $4.618 + 2.382 = 7.0$ recovers the trace.

For $\lambda_1 \approx 4.618$, solving $(A - \lambda_1 I)v = 0$ gives $v_1 \propto [1, 0.618]$; for $\lambda_2 \approx 2.382$, $v_2 \propto [-0.618, 1]$. Dot product: $1 \times (-0.618) + 0.618 \times 1 = -0.618 + 0.618 = 0$. Perpendicular, exactly as promised, with no arrangement needed. The diagram draws these two axes with their stretch labels.

Code

The point is the check: reconstruct from the factors and confirm orthonormality, with no explicit inverse anywhere.

1import numpy as np2
3
4A = np.array([[4.0, 1.0], [1.0, 3.0]])5w, Q = np.linalg.eigh(A)  # symmetric solver: orthonormal Q guaranteed6print(np.allclose(Q @ np.diag(w) @ Q.T, A))  # -> True7print(np.allclose(Q.T @ Q, np.eye(2)))  # -> True

eigh (the symmetric solver) returns real eigenvalues and orthonormal vectors by construction. The general eig would work here too but promises less; on symmetric input, always reach for eigh. The returned values are approximately $[2.382, 4.618]$ in ascending order.

Watch Out For

The Quick Version

• Symmetric matrices split into $Q\Lambda Q^T$ with real eigenvalues and perpendicular eigenvectors.
• The 4-1-1-3 example gives eigenvalues $4.618$ and $2.382$ with orthogonal axes $[1, 0.618]$ and $[-0.618, 1]$.
• Trace $7$ and determinant $11$ are recovered as sum and product, a free hand-check.
• The sum form shows $A$ as stacked rank-one projections, which is what PCA truncates.
• Use eigh, not eig, on symmetric input: orthonormality comes guaranteed.
• Eigenvector signs are arbitrary; compare subspaces, never raw vectors.

What to Read Next

• Eigenvalues and Eigenvectors
• Orthogonality and Projections