Image Stitching

Find the landmarks two photos share, keep only the matches that agree on one camera motion, and warp every frame onto a single canvas.

Stitching keeps the 140 inlier matches of 200 (70 percent) that agree on one homography and routes one seam through their overlap

Why Does This Exist?

No affordable sensor captures a mountain range or a whole factory floor in one frame, yet inspectors and hikers want the whole scene at full resolution. The obvious fix is laying frames side by side or averaging overlaps, and it breaks visibly: small misalignments double every edge into ghosts, and exposure differences draw a stripe at each boundary. Stitching manufactures wide views from overlapping narrow ones instead, keeping every pixel sharp rather than shrinking the world to fit. Phone panoramas, satellite mosaics, and microscope slide maps all run this pipeline.

Two prerequisites carry the load: repeatable landmarks from keypoint detection, and the warp math from transformations and homography. This page covers detection, matching, robust estimation, and compositing. The warp itself is estimated as in homography estimation and cleaned with RANSAC.

Think of It Like This

Tiling with overlapping transparencies

Print a map across overlapping transparencies and reassemble it by sliding sheets until roads continue unbroken across every overlap, then trim the doubled regions away. Shared landmarks are the road crossings, geometric agreement is the unbroken road, and trimming is compositing.

It stops holding when sheets bend differently. Transparencies that stretched in the heat never align everywhere at once, the same way parallax and lens distortion defeat a single global warp. Flat-planed, well-calibrated captures tile; violated assumptions wrinkle.

How It Actually Works

Detect, describe, match

Each frame yields keypoints with descriptors (SIFT or ORB classically) that survive rotation and exposure shifts. Nearest-neighbour matching in descriptor space proposes pairs, and Lowe's ratio test keeps only pairs whose best match clearly beats the runner-up, typically at a $0.75$ ratio. A run might propose $200$ raw matches with $140$ surviving as inliers later, a $70\%$ keep rate that signals solid overlap; under $20$ surviving inliers means rejection, not a panorama.

One motion from noisy votes

True matches all obey one geometric motion (a homography for planar scenes or pure rotation), while mismatches scatter. RANSAC samples minimal point sets, fits candidate motions, and keeps the motion with the most supporters, discarding the rest as outliers. Four points determine a homography, so even heavy mismatch rates resolve when the true motion commands a plurality.

Warp and composite

The winning homography warps each frame onto a shared canvas, usually anchored to a central reference frame to spread distortion. Overlapping pixels then need a seam decision (which frame owns each pixel) before blending, and the canvas crops to the valid region. Bundle adjustment refines all motions jointly for sequences longer than pairs, stopping drift from accumulating frame to frame.

Worked example

Take the run above: $200$ raw descriptor matches, of which $140$ support the winning homography. The keep rate is $140 / 200 = 0.70$ , or $70\%$ . With inlier ratio $w = 0.70$ and $n = 4$ points per homography sample, one random sample is all inliers with probability $w^n = 0.70^4 = 0.2401$ . The chance a sample is contaminated is $1 - 0.2401 = 0.7599$ . Over $k = 20$ independent samples the chance every sample is contaminated is $0.7599^{20} \approx 0.0041$ , so the chance at least one clean sample is drawn is $1 - 0.0041 = 0.9959$ , about $99.6\%$ . Twenty rounds are plenty at this inlier ratio; at $w = 0.30$ the same confidence would need hundreds.

Code

The point of this snippet is the arithmetic above, recomputed with only the standard library: keep rate plus the probability that $20$ RANSAC rounds find at least one clean $4$ -point sample at $w = 0.70$ .

1# -> Keep rate and RANSAC success odds for the worked example.2raw, inliers = 200, 1403keep = inliers / raw  # 140 / 2004w, n, k = 0.70, 4, 205p_clean_one = w**n6p_all_bad = (1 - p_clean_one) ** k7print(round(keep, 4), round(p_clean_one, 4), round(1 - p_all_bad, 4))8# -> (0.7, 0.2401, 0.9959)

A keep rate of $0.70$ gives $0.2401$ clean-sample odds per round and $0.9959$ success odds across $20$ rounds. Real stitchers adapt $k$ online as the inlier ratio estimate changes instead of fixing it at $20$ .

Watch Out For

Parallax from translating through close scenes

Symptom: near objects double or shear while distant mountains align perfectly, and no parameter fixes both at once. One homography models one plane or pure rotation; close foreground violates both. Shoot rotation-only captures about the lens nodal point, keep scenes distant, or mask moving foreground before matching.

Visible exposure steps between frames

Symptom: alignment is perfect but every frame boundary shows as a brightness stripe from auto-exposure or vignetting. Geometry succeeded and photometry failed. Fix: enable gain compensation and vignette correction before compositing, lock exposure across the sweep, and let multi-band blending dissolve whatever steps remain.

The Quick Version

Stitching chains detect, match, robustly estimate, warp, and composite across overlapping frames.
Ratio-tested descriptor matches plus RANSAC turn noisy pairs into one trustworthy camera motion.
Four points determine a homography, and $140$ of $200$ matches kept means a $70\%$ inlier ratio.
At $w = 0.70$ , $20$ RANSAC rounds find a clean sample with about $99.6\%$ probability ( $0.9959$ ).
Parallax and exposure steps are the two classic failures, needing capture discipline and photometric correction.