ML & Data Science Interviews · Evaluation Metrics · Lesson 2 of 3
ROC-AUC, PR curves & thresholds
Precision/recall depend on a chosen threshold. Threshold-free curves summarize a classifier across all thresholds so you can compare models before fixing an operating point.
- ROC curve: true-positive rate vs false-positive rate across thresholds. ROC-AUC = probability the model ranks a random positive above a random negative (0.5 = random, 1.0 = perfect).
- PR curve: precision vs recall across thresholds. PR-AUC (average precision) is far more informative than ROC-AUC under heavy class imbalance.
- ROC-AUC can look deceptively high on rare-positive problems because true negatives dominate the FPR denominator; PR curves stay honest.
- The curve chooses the model; the business cost chooses the operating threshold on it.
Imbalance? Prefer PR-AUCWith 0.1% positives, a model can have ROC-AUC of 0.95 and still be useless in production. PR-AUC focuses on the positive class and exposes that gap — mention it and you sound senior.
◆ Lock it in
- ROC-AUC = ranking quality across all thresholds; 0.5 random, 1.0 perfect.
- Under heavy imbalance, prefer PR-AUC — ROC-AUC can flatter a bad model.
- The curve picks the model; the cost of errors picks the threshold.
Feynman drill — say it out loudExplain why PR-AUC is often more informative than ROC-AUC on a rare-event problem.
Step 1 rate your confidence · Step 2 pick your answer
For a dataset with 0.5% positive cases, which evaluation is generally most informative?
Step 1 — how sure are you?