Template Matching

Template matching finds an object by sliding its picture across the image like a stencil and keeping the spot where the pixels line up best.

A template patch scores every image position by correlation, and the score map peaks at 1.0 exactly where the patch reappears.

Why Does This Exist?

Sometimes the thing you want to find looks almost exactly like the picture you already have: a logo on a scanned form, a button in a screenshot test, one frame's patch in the next frame. Training a classifier for that is overkill. You have the pixels, so just look for those pixels.

Template matching is that direct search, and it is the baseline every fancier classical detector had to beat. It also teaches the core pain of the whole family: sliding a window everywhere is simple, correct, and brutally expensive the moment the object rotates, scales, or the light changes. When fixed patches stop coping, the answer is descriptors that survive those changes, starting with feature matching.

This page covers the normalized score, a hand-computed 2 by 2 example, and why peaks need thresholds.

Think of It Like This

A stencil over newsprint

Cut a word out of a newspaper, then slide the cut-out hole over every column of another page, checking at each stop how well the letters underneath align with the hole. Where the text matches, words show through cleanly. Everywhere else you get fragments.

The cut-out is the template, the sliding is the search, and showing through cleanly is the correlation score. And the failure is built in: shrink the newspaper on a copier and the stencil never fits again, because the hole is a fixed size.

How It Actually Works

The normalized score

Place the $m \times n$ template over the image at position $(x, y)$ and compare the template pixels $T$ against the covered image patch $I$ . The workhorse is normalized cross-correlation:

R(x, y) = \frac{\sum (T - \bar{T})(I - \bar{I})}{\sqrt{\sum (T - \bar{T})^2 \sum (I - \bar{I})^2}}

Here $\bar{T}$ and $\bar{I}$ are the mean brightness of template and patch, so subtracting them makes the score ignore uniform lighting shifts. The result runs from $-1$ (inverted) to $1$ (perfect match). The output is a whole score map, and the best match is its peak.

Worked 2 by 2 example

Take a 2 by 2 template $\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$ . Against an identical patch, both means are $2.5$ , the numerator is $(-1.5)^2 + (-0.5)^2 + 0.5^2 + 1.5^2 = 2.25 + 0.25 + 0.25 + 2.25 = 5$ , each denominator sum is $5$ , and $R = 5 / \sqrt{5 \times 5} = 1.0$ . Against the flipped patch $\begin{bmatrix} 4 & 3 \\ 2 & 1 \end{bmatrix}$ every deviation changes sign, so the numerator becomes $-5$ and $R = -1.0$ . Same edges, opposite polarity, and the score says so exactly.

Peaks need thresholds

The naive rule takes the argmax of the score map. In practice set a threshold such as $0.8$ and treat everything below as not found, because a peak among garbage is still garbage. Rotation, scale change, and occlusion all flatten the true peak, which is why production use pairs matching with image pyramids and banks of rotated templates, each multiplying the cost.

Code

1def ncc(template: list, patch: list) -> float:2    """Normalized cross-correlation between two equal-size grids."""3    t = [v for row in template for v in row]4    p = [v for row in patch for v in row]5    tm = sum(t) / len(t)6    pm = sum(p) / len(p)7    td = [v - tm for v in t]8    pd = [v - pm for v in p]9    num = sum(a * b for a, b in zip(td, pd))10    den = (sum(a * a for a in td) * sum(b * b for b in pd)) ** 0.511    return round(num / den, 2)12
13
14t = [[1, 2], [3, 4]]15print(ncc(t, t))16# -> 1.017print(ncc(t, [[4, 3], [2, 1]]))18# -> -1.019print(ncc(t, [[1, 2], [3, 5]]))20# -> 0.98

Watch Out For

Bright patches win for free

Without normalization, raw correlation and sum-of-squared-differences both favor bright image regions regardless of content. A white wall outscores a dim true match.

Always use the normalized score above, and distrust any tutorial version that skips the mean subtraction.

One template, one size, one angle

A template matches itself at its own scale and rotation and little else. Covering variation needs a pyramid of scales plus rotated copies, and the cost multiplies with each.

When the template bank starts looking like a training set, switch to a feature method such as Haar cascades or learned descriptors.

The Quick Version

Template matching correlates a patch against every image position and takes the score peak.
Normalized cross-correlation runs from $-1$ to $1$ and ignores uniform brightness shifts.
The identical 2 by 2 patch scores 1.0; its flipped twin scores $-1.0$ ; a near miss scores 0.98.
Rotation, scale, and occlusion flatten the peak, so real use needs pyramids and template banks.
Cheap, exact, and brittle: the baseline classical detectors had to beat.