ML & Data Science Interviews · Evaluation Metrics · Lesson 3 of 3
Regression metrics & handling class imbalance
For continuous targets you measure how far off you are; for skewed classes you fix the data or the loss so the minority class isn't ignored.
Regression metrics
- MAE (mean absolute error): average absolute miss, in target units, robust to outliers.
- RMSE (root mean squared error): squares errors first, so it punishes large errors more; same units as target.
- R-squared: fraction of variance explained (1 = perfect, 0 = no better than predicting the mean, can go negative).
Handling class imbalance
- Reweight first: class weights in the loss so minority errors cost more — cheap, and it doesn't distort the data.
- Right metric + threshold: use F1/PR-AUC, and tune the decision threshold — not 0.5.
- Resample as a fallback: oversample the minority (e.g. SMOTE) or undersample the majority — but resampling skews predicted probabilities, so recalibrate if you need them. Leading with SMOTE reads dated in 2026.
- Never balance the test set — it must reflect the real distribution.
RMSE and outliersBecause RMSE squares errors, a few large outliers dominate it. If you don't want big misses to swamp the metric (or your business tolerates them), report MAE too — pick the one whose error-shape matches the real cost.
◆ Lock it in
- MAE robust and in-units; RMSE punishes big errors; R-squared = variance explained.
- Fight imbalance with class weights + the right metric/threshold first; resampling is the fallback (and skews calibration).
- Keep the test set at the true distribution.
Feynman drill — say it out loudExplain when you'd report MAE instead of RMSE, and name two ways to handle a 1:100 class imbalance.
Step 1 rate your confidence · Step 2 pick your answer
Which regression metric most heavily penalizes a few large prediction errors?
Step 1 — how sure are you?