Harris Corner Detector Finds Stable Points

Harris turns every pixel into a corner score from local gradients, so tracking keeps points that stay put when the camera shifts.

Gradients feed a 2 by 2 structure tensor and the response R equals 17.25 for the corner window but minus 0.17 for the edge window

Why Does This Exist?

Tracking and stitching need points that can be found again. Flat patches all look alike, and edges slide along their own direction, so neither survives a viewpoint change. Corners lock in two directions at once, which makes them the only local features stable under small camera motion. Harris gave this a cheap test that uses only first derivatives in a small window, with no eigenvalue solver and one response value per pixel. Read the overview in corner and keypoint detection first, then come back for the exact score; when scale also changes, move up to scale invariant features.

Think of It Like This

Tent stakes in the wind

Picture a tent stretched on a lawn. Push the middle of a flat sheet and you cannot tell where you pushed. Push along a seam and the fabric slides sideways. Pull the corner stake and the whole shape answers at once, because the stake constrains every direction.

Harris looks for tent stakes in the image. Flat means no pull in any direction, edge means pull in one direction, corner means pull in every direction. Where it stops: pixels do not stretch like fabric, so only gradient energy counts and a zoomed view breaks the verdict.

How It Actually Works

Start with horizontal and vertical derivatives $I_x$ and $I_y$ at each pixel, usually from Sobel filters, then form the structure tensor over a Gaussian window $w$ around each pixel:

$M = \sum w \begin{bmatrix} I_x^2 & I_x I_y \\ I_x I_y & I_y^2 \end{bmatrix}$

$M$ summarizes gradient energy in the neighborhood. Let its eigenvalues be $\lambda_1$ and $\lambda_2$ : both small means flat, one large and one small means edge, both large means corner. Eigenvalues cost too much per pixel, so Harris uses the determinant and trace shortcut with $k$ near $0.04$ :

$R = \det(M) - k \cdot \mathrm{trace}(M)^2$

Here $\det(M) = \lambda_1 \lambda_2$ and $\mathrm{trace}(M) = \lambda_1 + \lambda_2$ . Threshold $R$ and keep local maxima through non-maximum suppression. Harris is rotation invariant because rotating the image rotates both eigenvectors together, but it is not scale invariant since a zoom turns the corner into a curve.

Worked example

Take a window where $\sum I_x^2 = 5.0$ , $\sum I_y^2 = 4.2$ , and $\sum I_x I_y = 0.6$ . Then $\det = 5.0 \times 4.2 - 0.6^2 = 21.0 - 0.36 = 20.64$ and $\mathrm{trace} = 5.0 + 4.2 = 9.2$ with $9.2^2 = 84.64$ . With $k = 0.04$ , the penalty is $0.04 \times 84.64 = 3.39$ and $R = 20.64 - 3.39 = 17.25$ , a large positive value that marks a corner. An edge window with sums $5.0$ , $0.2$ , and $0.3$ gives $\det = 5.0 \times 0.2 - 0.09 = 0.91$ , $\mathrm{trace} = 5.2$ with $5.2^2 = 27.04$ , penalty $0.04 \times 27.04 = 1.08$ , and $R = 0.91 - 1.08 = -0.17$ , so it is rejected.

Code

The point of this snippet is the response shortcut, not the filtering: it reproduces the 17.25 corner and the minus 0.17 edge the prose derived by hand.

1def harris_r(sxx, syy, sxy, k=0.04):2    det = sxx * syy - sxy * sxy3    trace = sxx + syy4    return det - k * trace * trace5
6print(round(harris_r(5.0, 4.2, 0.6), 2))7# -> 17.258print(round(harris_r(5.0, 0.2, 0.3), 2))9# -> -0.17

Positive large means corner, negative means edge, near zero means flat. Real pipelines feed these scores into thresholding plus local maxima.

Watch Out For

Threshold picked on one image

A threshold that looks clean on a textured poster fires everywhere on gravel and nowhere on soft faces. The symptom is corner counts that swing by orders of magnitude across lighting. Tune $k$ and the threshold on your own contrast range, then add non-maximum suppression, since a raw threshold without local maxima gives clusters instead of points.

Expecting scale invariance

Harris has no pyramid, so matching a close-up against a wide shot fails. The symptom is tracking that dies the moment the camera dollies in. Fix it by moving to scale invariant features or a pyramid detector instead of raising the threshold.

The Quick Version

Harris builds a $2 \times 2$ structure tensor from $I_x$ and $I_y$ in each window.
Response $R = \det(M) - k \cdot \mathrm{trace}(M)^2$ avoids an eigenvalue solver.
The fixture scores $R = 17.25$ for the corner and $R = -0.17$ for the edge.
Rotation invariant but scale sensitive, so it suits tracking at fixed distance.
Threshold plus non-maximum suppression turns scores into usable points.