Homography Estimation Fits Planes

Homography estimation solves the 3 by 3 matrix that maps one plane to another, so tilted documents flatten and photo pairs align.

Four tilted corners pin to a 300 by 300 square through an 8 by 9 DLT system, and a fifth pair reprojects within 2 pixels to count as an inlier

Why Does This Exist?

Two photos of one flat surface differ by perspective, not just shift and zoom. Posters, documents, courts, and walls need a plane to plane map before they can merge or be measured. The $3 \times 3$ homography $H$ is that map. Estimation turns matched keypoints into $H$ with the Direct Linear Transform inside RANSAC, then warps with it for stitching. For the transform basics first, read image transformations and homography.

Think of It Like This

Pinning a poster flat

A poster photographed at an angle looks like a trapezoid. Pin its four corners onto a rectangular frame and the print flattens, with each pin constraining the stretch a little more. Point pairs are those pins: four good pins define the warp and extra pins steady it against noise.

Where it stops: pins assume one flat sheet, so curved pages or scenes with real depth need richer models than a single homography.

How It Actually Works

A homography maps homogeneous points $x$ to $x'$ up to scale:

$x' \sim H x, \quad H = \begin{bmatrix} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & 1 \end{bmatrix}$

Fixing $h_{33} = 1$ leaves eight degrees of freedom, so at least four point pairs are needed. The Direct Linear Transform builds two rows of a matrix $A$ per pair from the cross product constraint $x' \times Hx = 0$ and solves $Ah = 0$ by SVD. Normalize points into $[-1, 1]$ first for stable math, then denormalize the result. RANSAC wraps DLT: sample four pairs, score every pair by reprojection error, keep the model with the most inliers under the pixel threshold, and refit $H$ on all inliers. Warp with inverse sampling plus bilinear interpolation for clean output.

Worked example

Map source corners $(56, 65)$ , $(368, 52)$ , $(28, 387)$ , $(389, 390)$ to the square $(0, 0)$ , $(300, 0)$ , $(0, 300)$ , $(300, 300)$ . Four pairs give two rows each, hence an $8 \times 9$ matrix $A$ , and SVD returns the 9-vector $h$ up to scale with $h_{33} = 1$ . A fifth pair at $(200, 200)$ mapping near $(150, 155)$ then reprojects within $2$ pixels under the fitted $H$ , so it counts as an inlier at a threshold of $3$ . The final $H$ refits on all inliers and warps source to destination. Note the pairs must spread over the plane: four collinear points leave perspective underdetermined and DLT returns a wild warp.

Code

The point of this snippet is the apply-and-check step, not the SVD: it maps a point through $H$ in homogeneous coordinates and measures the reprojection error that decides inliers.

1def apply_h(H, x, y):2    xp = H[0][0] * x + H[0][1] * y + H[0][2]3    yp = H[1][0] * x + H[1][1] * y + H[1][2]4    w = H[2][0] * x + H[2][1] * y + H[2][2]5    return (xp / w, yp / w)6
7H = [[2.0, 0.0, 10.0], [0.0, 2.0, 20.0], [0.0, 0.0, 1.0]]8print(apply_h(H, 5.0, 5.0))9# -> (20.0, 30.0)10
11def reproj_error(H, x, y, xp, yp):12    qx, qy = apply_h(H, x, y)13    return ((qx - xp) ** 2 + (qy - yp) ** 2) ** 0.514
15print(round(reproj_error(H, 5.0, 5.0, 20.0, 30.0), 2))16# -> 0.017print(reproj_error(H, 5.0, 5.0, 21.5, 30.0) < 3.0)18# -> True

The exact pair gives zero error, and a prediction $1.5$ pixels off still counts as an inlier at threshold $3$ , exactly the rule the worked fifth pair passes.

Watch Out For

Solving from collinear points

Four points on one line leave perspective underdetermined, and DLT returns wild warps. The symptom is a matrix that flattens everything into streaks. Check point spread and reject samples with tiny enclosed area before solving.

Using one homography off the plane

$H$ models one plane or pure rotation only, so foreground chairs against a back wall break it. The symptom is a wall that aligns while every chair ghosts double. Segment planes or move to epipolar geometry when depth varies.

The Quick Version

A homography has eight degrees of freedom with $h_{33}$ fixed to one.
Four point pairs give a minimal DLT solution through an $8 \times 9$ matrix $A$ .
Normalize points before SVD for stable results.
RANSAC plus a reprojection threshold near $3$ pixels handles outliers.
Warp with inverse mapping and interpolation for clean output.