SIFT Scale Invariant Features

SIFT finds distinctive points that survive zoom and rotation, then fingerprints each one with a 128-number vector, so two photos of the same place still match from different viewpoints.

SIFT detects scale-space extrema, fixes each point's compass bearing, and pools gradients into a 128-number fingerprint that matches across zoom and rotation.

Why Does This Exist?

The Harris corner detector finds corners at one fixed scale: photograph the same plaza from twice as far away and the corners it learned vanish, because the window that held a corner now holds an edge. Raw patch matching has the mirror flaw: rotate the camera 35 degrees and every pixel template stops lining up. Real pairs mix both, a tourist snapshot against a wide panorama frame in image stitching, so matching needed points that survive zoom plus rotation plus moderate lighting change in one pipeline.

SIFT (David Lowe, 1999 to 2004) chains four ideas to get there: Difference of Gaussians extrema for scale, subpixel refinement plus edge rejection for stability, orientation voting for rotation, and pooled gradient histograms for identity. That chain made panorama stitching and instance retrieval practical years before learned features. This page covers all four stages with the thresholds and the 128-number construction; how pairs of descriptors get compared lives in feature matching, and the faster binary alternative is ORB.

Think of It Like This

A surveyor mapping lone oak trees

A surveyor must map lone oaks so a colleague can re-find them from any aircraft altitude. First she spots candidate trees that stand out at every altitude, not just one. Then she records each tree's lean as a compass bearing, so the map has its own north per tree. Then she sketches the branch pattern inside sixteen small squares around each trunk and files the sketch.

Spotting across altitudes is DoG extrema detection, the compass bearing is orientation assignment, and the sixteen-square sketch is the descriptor. The analogy stops at seasons: the surveyor still recognizes a winter oak, but SIFT's gradient sketch degrades under day-night change and fails at extreme perspective, where the pattern itself is foreshortened away.

How It Actually Works

Finding points that survive zoom

Blur the image with Gaussians at increasing $\sigma$ (sigma, the blur radius) and subtract adjacent blurs to get Difference of Gaussians images $D(x, y, \sigma)$ . A keypoint candidate is a pixel that is a maximum or minimum among its 26 neighbors: 8 in its own scale slice plus 9 in the slice above plus 9 below. Each extremum is refined to subpixel accuracy with a Taylor fit, then two rejections apply: points with $|D| < 0.03$ are low contrast and dropped, and points on edges are dropped when the Hessian eigenvalue ratio $r$ exceeds 10, since edges slide along themselves and never localize.

Giving each point its own north

Around each survivor, collect gradient directions into a 36-bin orientation histogram, so each bin spans 10 degrees. The tallest bin is the dominant orientation; any other bin above 80 percent of the peak spawns an extra keypoint at the same location with that orientation. Say the peak sits at 35 degrees: the sampling grid for everything after is rotated by minus 35 degrees, so the description that follows is computed in the point's own frame and rotation of the camera cancels out.

The 128-number fingerprint

Take the rotated $16 \times 16$ window, split it into 16 cells of $4 \times 4$ pixels, and vote each cell's gradients into 8 orientation bins. That yields $16 \times 8 = 128$ floats. Normalize the vector to unit length, clamp every value at 0.2, and renormalize: the clamp caps the influence of any single strong edge, which is what buys robustness to nonlinear lighting like a shadow edge crossing the patch.

Worked numbers

Suppose one cell holds gradient mass $[0, 2, 9, 4, 1, 0, 0, 0]$ across its 8 bins. Bin three holds 9 of the 16 total mass, so that direction dominates 8 of the final 128 numbers. After normalization, a true pair of views of this point lands at Euclidean distance 0.0424 while the nearest wrong candidate sits at 0.7599, giving a nearest-to-second-nearest ratio of $0.0424 / 0.7599 = 0.0558$ . The standard ratio test accepts matches below 0.8, so this pair passes with wide margin, and the same numbers run in the snippet below.

Code

1import math2
3def euclid(a: list[float], b: list[float]) -> float:4    """Euclidean distance between two descriptor vectors."""5    return math.sqrt(sum((x - y) ** 2 for x, y in zip(a, b)))6
7query = [0.31, 0.12, 0.55, 0.40]8near = [0.33, 0.10, 0.52, 0.41]9far = [0.05, 0.71, 0.22, 0.63]10d1, d2 = euclid(query, near), euclid(query, far)11print(round(d1, 4), round(d2, 4), round(d1 / d2, 4))12# -> 0.0424 0.7599 0.055813print(d1 / d2 < 0.8)14# -> True

Watch Out For

Leaving octave doubling on for thumbnails

SIFT doubles the input image by default to catch fine detail. On small thumbnails or downscaled video frames that just magnifies blur and multiplies junk points along compression artifacts. The symptom is thousands of keypoints with near-random matches. Disable the doubling or raise the contrast threshold when the input is already small.

Expecting frame rate on CPU

A 128-float descriptor per point plus a full DoG pyramid costs real milliseconds per frame, so SIFT suits offline stitching and retrieval, not live tracking. The symptom is a tracker that works on clips but stalls on a live feed. For live use, switch to ORB or a learned binary descriptor and keep SIFT for the map-building side.

The Quick Version

DoG extrema across 26 neighbors plus subpixel refinement give repeatable points across scale.
Contrast below $|D| = 0.03$ and edge ratios above $r = 10$ are rejected as unstable.
One 36-bin vote per point gives rotation invariance; peaks above 80 percent spawn extra keypoints.
Sixteen $4 \times 4$ cells times 8 bins give the 128-float descriptor, clamped at 0.2.
Nearest-to-second-nearest distance ratio below 0.8 accepts the match; the worked pair scores 0.056.