Tracking Metrics MOTA IDF1 HOTA

Tracking metrics split honesty three ways, asking whether the tracker found everything, kept every identity, and drew tight boxes.

One miss and one false alarm over ten boxes give MOTA 0.80 while perfect identities keep IDF1 at 0.90, so each score sees half the story.

Why Does This Exist?

A tracker can find every pedestrian yet scramble every name, or hold names perfectly while missing half the crowd. One score cannot praise and blame both halves, so comparing SORT against DeepSORT on a single number misleads. Tracking metrics exist as a trio that splits the credit: MOTA for detection hygiene, IDF1 for identity consistency, and HOTA for their balance. Teams that tune to one of them ship the failure the others would have caught, which is why benchmarks report all three alongside raw switch counts.

This page covers what each metric counts, a hand-scored ten-box example, and which knob moves which score.

Think of It Like This

Grading an exam in three passes

An exam gets three passes: attendance (every question attempted), name tags (each answer under the right heading), and neatness (boxes drawn tightly). MOTA grades attendance by counting blanks, scribbles and swapped tags. IDF1 grades the tags globally across the whole paper. HOTA multiplies neatness by attendance per answer, then averages. A student can ace attendance while failing tags, exactly like a detector-heavy tracker with sparkling MOTA and scrambled identities.

The analogy stops at the paper's edge. Exam grading is final, while tracking metrics shift with the matching threshold and annotation rules, so scores only compare runs graded identically.

How It Actually Works

MOTA counts detection errors

Let GT be the ground-truth box count, FN the misses, FP the false alarms, and IDSW the identity switches. Then:

$\text{MOTA} = 1 - \frac{\text{FN} + \text{FP} + \text{IDSW}}{\text{GT}}$

MOTA is detection-heavy: one switch costs the same as one miss, so association failures barely dent it. It can go negative on terrible runs, which dashboards must show rather than clamp.

IDF1 scores identity consistency

IDF1 is the F1-score of identity matches. Each predicted identity maps to exactly one true identity across the whole sequence (global, not per-frame), giving identity true positives IDTP, false positives IDFP and false negatives IDFN:

$\text{IDF1} = \frac{2 \cdot \text{IDTP}}{2 \cdot \text{IDTP} + \text{IDFP} + \text{IDFN}}$

Because the mapping is global, one person tracked under two names scores badly no matter how tight the boxes are. Association fixes move IDF1; detector fixes barely touch it.

HOTA balances both

HOTA is the geometric mean of detection accuracy and association accuracy, averaged over matched pairs: $\text{HOTA} = \sqrt{\text{DetA} \cdot \text{AssA}}$ . DetA rewards finding boxes with tight overlap, AssA rewards keeping their names across time. A tracker must do both to score well, which is why HOTA is the default headline on current benchmarks.

Worked ten-box trio

Ten ground-truth boxes produce 1 miss, 1 false alarm and 0 switches. MOTA $= 1 - (1 + 1 + 0)/10 = 0.80$ . Identity mapping gives IDTP $= 9$ , IDFP $= 1$ , IDFN $= 1$ , so IDF1 $= 18/20 = 0.90$ : association is healthy while detection leaks. Detection accuracy is $9/11 \approx 0.818$ and association is near perfect, so HOTA $\approx \sqrt{0.818} \approx 0.90$ . A rival run with 0 detection errors but 4 switches flips the story: MOTA drops to 0.60 while IDF1 collapses, correctly blaming association instead.

Code

1def mota(misses, false_alarms, switches, ground_truth):2    return 1 - (misses + false_alarms + switches) / ground_truth3
4print(round(mota(1, 1, 0, 10), 2))5# -> 0.86print(round(mota(0, 0, 4, 10), 2))7# -> 0.6

Watch Out For

Tuning to MOTA while identities burn

MOTA charges one penalty per switch, the same as one miss, so teams optimize the detector and declare tracking solved. The symptom is leaderboard MOTA beside users complaining that counted people change numbers mid-store. Gate releases on IDF1 and raw switch counts, never on MOTA alone.

Clamping a negative MOTA to zero

MOTA goes negative when errors outnumber ground-truth boxes, and dashboards that floor it at zero make a catastrophic tracker look tied with a mediocre one. The symptom is two flat zero rows hiding very different failure sizes. Keep the raw value so the distance to usable stays visible.

The Quick Version

MOTA $= 1 - (\text{misses} + \text{false alarms} + \text{switches}) / \text{ground truth}$ , and is detection-heavy.
IDF1 is the F1-score under one global identity mapping, and is association-heavy.
HOTA $= \sqrt{\text{DetA} \cdot \text{AssA}}$ balances tight boxes with kept names.
Ten boxes with 1 miss and 1 false alarm score MOTA 0.80 with IDF1 0.90.
Report all three plus raw switch counts, or single-metric tuning will mislead.