SSD Object Detector

SSD predicts boxes from several feature-map scales in one pass, so shallow layers catch small objects while deep layers catch large ones.

SSD tiles anchors across six shrinking feature maps at once, so a small anchor on a fine map claims a distant pedestrian while a large anchor on a coarse map claims a nearby bus.

Why Does This Exist?

Early single-shot detectors predicted from one coarse grid, so small objects blurred into single cells and vanished. The fix could have been an image pyramid, running the whole network per scale, but that multiplies cost per scale. SSD, published by Liu and coauthors in 2016, took the cheaper road: run the backbone once, then attach predictors to six feature maps of shrinking resolution inside that single pass. Fine maps with small anchor boxes handle tiny objects, coarse maps with large anchors handle big ones, and the network reached 59 frames per second while matching two-stage accuracy on standard benchmarks.

The pieces around it each have their own page: anchor boxes for the prior-box mechanism, YOLO for the sibling one-stage lineage, non-maximum suppression for the duplicate cleanup SSD relies on, and RetinaNet for the successor that replaced hard-negative mining with focal loss. This page covers the multi-scale heads, the matching rule with a hand-scored IoU example, and the mining ratio that keeps training balanced.

Think of It Like This

Casting six net sizes in one throw

A fisher with one net catches one fish size per trip. SSD casts six nets at once: a fine mesh near the surface for minnows, progressively wider meshes deeper for bigger fish, all hauled in together.

Each feature-map scale is one net and each anchor size is its mesh. One cast covers the whole size range in a single pass.

The analogy stops at the water. Real nets never overlap, while neighboring SSD anchors do, so overlapping claims on the same fish need settling afterward, which is the job of non-maximum suppression.

How It Actually Works

Multi-scale heads and matching

Extra convolutional layers extend the backbone into a descending staircase, for example $38 \times 38$ down to $1 \times 1$ across six levels. Each level tiles anchors sized to its resolution and predicts class scores plus box offsets directly from every anchor. During training, each ground-truth box matches every anchor with intersection over union above $0.5$ , plus the single best anchor regardless of threshold, guaranteeing each object supervises at least one predictor. Here IoU is the overlap area divided by the union area of the two boxes.

Worked example: one match, one miss

Score two $10 \times 10$ anchors against the ground-truth box $(5, 5, 15, 15)$ , in $(x_1, y_1, x_2, y_2)$ form. Anchor A is $(0, 0, 10, 10)$ : overlap spans $5 \times 5 = 25$ , union is $100 + 100 - 25 = 175$ , so IoU $= 25/175 \approx 0.1429$ , below $0.5$ , no match. Anchor B is $(4, 4, 14, 14)$ : overlap spans $9 \times 9 = 81$ , union is $100 + 100 - 81 = 119$ , so IoU $= 81/119 \approx 0.6807$ , above $0.5$ , a match that supervises this object. One grid cell away separates supervision from silence, which is why anchor scales must blanket the object sizes in the data.

Hard-negative mining

Tens of thousands of anchors per image make negatives outnumber positives roughly $100$ to $1$ . SSD sorts negative boxes by confidence loss and keeps only the top three per positive, a $3$ to $1$ cap that keeps the gradient honest. This explicit mining predates focal loss and solves the same drowning problem by discarding the easy background the network already masters.

Code

1def iou(a, b):2    """Intersection over union for boxes in (x1, y1, x2, y2) form."""3    ix = max(0, min(a[2], b[2]) - max(a[0], b[0]))4    iy = max(0, min(a[3], b[3]) - max(a[1], b[1]))5    inter = ix * iy6    union = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - inter7    return inter / union8
9truth = (5, 5, 15, 15)10for name, box in [("A", (0, 0, 10, 10)), ("B", (4, 4, 14, 14))]:11    score = iou(box, truth)12    print(name, round(score, 4), "match" if score > 0.5 else "miss")13# -> A 0.1429 miss14# -> B 0.6807 match

Watch Out For

Anchor scales copied blindly to a new dataset

Default scales assume everyday photo object sizes. On aerial imagery where everything is tiny, most ground-truth boxes match nothing and recall collapses at the matching step, before any learning happens. Verify match coverage per scale on your own boxes first, and rescale the anchor sizes until every size band finds partners.

Expecting successor-level dense scenes

SSD predates focal loss and feature pyramids, so crowded small-object scenes still favor its successors. The symptom is merged or dropped boxes in crowds that RetinaNet separates. Pick SSD for speed on moderate scenes, not for dense crowds.

The Quick Version

SSD predicts from six feature-map scales in one forward pass at 59 frames per second.
Fine maps with small anchors catch small objects while coarse maps catch large ones.
Anchors above IoU $0.5$ match, and the best anchor matches regardless of threshold.
Hard-negative mining caps negatives at $3$ per positive against $100$ -to- $1$ odds.
Anchor scales must be re-tuned per dataset or small objects match nothing.