Visual explainer

Confusion Matrix

The 2x2 table behind every classification metric, worked with 40, 10, 20 and 30 calls.

A model that predicts all negative scores 0.90 accuracy while catching zero of ten positives.

Evaluating classifiers with a single aggregate metric creates dangerous blind spots. On an imbalanced dataset containing $90$ negative cases and $10$ rare positive targets, a dummy model predicting "negative" for every sample achieves a seemingly impressive $0.90$ ( $90\%$ ) accuracy. Yet its recall on the target class is exactly $0.00$ —it catches zero of ten critical events. Evaluating performance requires inspecting the underlying distribution of decisions.

Rows Predict, Columns Confirm

Rows hold what you predicted and columns hold what was actually true.

The confusion matrix organizes every classification decision into a two-dimensional grid: rows indicate model predictions while columns represent ground truth:

True Positive (TP): Predicted positive and actually positive (correct alarm).
False Positive (FP): Predicted positive but actually negative (Type I error / false alarm).
False Negative (FN): Predicted negative but actually positive (Type II error / missed target).
True Negative (TN): Predicted negative and actually negative (correct rejection).

Diagonal cells capture correct decisions; off-diagonal cells isolate distinct failure modes.

Four Buckets, One Hundred

Forty true positives, ten false alarms, twenty misses and thirty true negatives fill the grid.

Evaluating $100$ predictions against ground truth produces four distinct counts:

$\text{TP} = 40$
$\text{FP} = 10$
$\text{FN} = 20$
$\text{TN} = 30$

Marginal sums verify total consistency:

Predicted rows: $40 + 10 = 50$ positive calls, and $20 + 30 = 50$ negative calls.
Actual columns: $40 + 20 = 60$ actual positives, and $10 + 30 = 40$ actual negatives.
Grand total: $50 + 50 = 60 + 40 = 100$ .

Overall accuracy is $\frac{\text{TP} + \text{TN}}{N} = \frac{40 + 30}{100} = 0.70$ .

Precision Splits From Recall

Precision reads 40 over 50 for 0.80 and recall reads 40 over 60 for 0.667.

Precision and recall evaluate the same $40$ true positives against different reference totals:

$\text{Precision} = \frac{\text{TP}}{\text{TP} + \text{FP}} = \frac{40}{40 + 10} = \frac{40}{50} = 0.80$

$\text{Recall} = \frac{\text{TP}}{\text{TP} + \text{FN}} = \frac{40}{40 + 20} = \frac{40}{60} \approx 0.667$

Precision reads across the predicted row to measure alarm reliability: $80\%$ of declared positives were genuine. Recall reads down the ground-truth column to measure completeness: the model discovered $66.7\%$ of all real targets in the population.

F1 Leans Low

F1 blends 0.80 and 0.667 into 0.727, just above the flat 0.70 accuracy.

Combining precision and recall requires the harmonic mean ( $F_1$ -score), which penalizes uneven performance:

$F_1 = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} = 2 \times \frac{0.80 \times 0.6667}{0.80 + 0.6667} = \frac{1.0667}{1.4667} \approx 0.727$

An arithmetic average would yield $0.733$ . The harmonic formulation pulls the blended score down toward the lagging metric ( $0.667$ ), ensuring that neither high precision nor high recall can artificially inflate overall performance.

Match Costlier Error

A strict cutoff favors precision and a lax cutoff favors recall, so match the costlier error.

Operating a probabilistic classifier requires choosing a decision cutoff $\theta \in [0, 1]$ . Adjusting $\theta$ trades false alarms against missed detections:

Lax threshold ( $\theta = 0.20$ ): Maximizes recall ( $0.95$ ) by declaring positives aggressively. Essential in cancer screening and fraud prevention, where a false negative is catastrophic.
Strict threshold ( $\theta = 0.80$ ): Maximizes precision ( $0.95$ ) by demanding strong certainty before firing. Essential in spam filtering and automated trading, where false alarms carry severe operational costs.

Threshold selection must be calibrated to the real-world asymmetric cost of each error type.

Where It Breaks

With three classes the six off diagonal cells name exactly who gets confused with whom.

Expanding to $K$ classes produces a $K \times K$ matrix with $K$ diagonal elements and $K(K - 1)$ off-diagonal error cells. Across three classes (Cat, Dog, Bird):

Diagonal entries capture $50 + 45 + 28 = 123$ correct classifications.
Six off-diagonal cells pinpoint exact pairwise confusion: $7$ dogs were misclassified as cats, while $6$ birds were misclassified as dogs.

Rather than retraining indiscriminately, inspecting specific off-diagonal cells directs targeted feature engineering and data collection to resolve specific confusion pairs.

The Quick Version

Accuracy of 0.90 masks complete failure when minority classes are ignored.
Predicted labels form rows; actual ground-truth classes form columns.
The 100-sample fixture splits into 40 TP, 10 FP, 20 FN, and 30 TN.
Precision (row read) reaches 0.80; recall (column read) reaches 0.667.
Harmonic F1 sits at 0.727, pulling downward toward the lower recall value.
Adjust the decision threshold to penalize whichever error carries the higher cost.
Off-diagonal cells in multi-class matrices identify exact pairwise failure patterns.