Skip to content
AI360Xpert
Core ML
Visual explainer

ROC and AUC

Ranking quality visualised — why AUC survives class imbalance where accuracy fails.

Why accuracy hides the threshold choice
Why accuracy hides the threshold choice

A classification model usually outputs a score, not a hard label. Turning that score into a decision requires a threshold. Accuracy measures just one threshold; it doesn't describe the model's underlying capability.

Drawing the ROC Curve

TPR vs FPR across every threshold: the ROC curve drawn
TPR vs FPR across every threshold: the ROC curve drawn

The ROC curve plots the True Positive Rate against the False Positive Rate for every possible threshold. A point on the curve represents exactly one threshold choice.

Area Under the Curve (AUC)

AUC as ranking quality: random = 0.5, perfect = 1.0
AUC as ranking quality: random = 0.5, perfect = 1.0

AUC summarises the curve into a single number from 0 to 1. It measures the probability that a randomly chosen positive example gets a higher score than a randomly chosen negative one. It evaluates ranking, not scaling.

ROC vs Precision-Recall

ROC vs precision-recall: which to use under imbalance
ROC vs precision-recall: which to use under imbalance

When negatives vastly outnumber positives (e.g. fraud detection), the ROC curve can look deceptively good because true negatives inflate its denominator. Precision-Recall curves are more honest for imbalanced data.

Where It Breaks

The failure: AUC can be high when the top-ranked slice is still bad
The failure: AUC can be high when the top-ranked slice is still bad

A model can have a high overall AUC but still perform poorly at the extreme top of the ranking. If your business process only reviews the top 1% of scores, a great overall AUC is useless if that top 1% is full of false positives.

The Quick Version

  • Thresholds: Accuracy only evaluates a single cutoff point.
  • ROC: Visualises the tradeoff between finding positives and making false alarms across all cutoffs.
  • AUC: Measures pure ranking ability, independent of the threshold.
  • Imbalance: ROC is too optimistic on rare events; use PR instead.
  • Failure: AUC measures the whole curve, but you often only care about the very top.

What to Read Next