ML & Data Science Interviews · ML Fundamentals · Lesson 1 of 3
The bias–variance tradeoff
Generalization error decomposes into three parts: bias (error from wrong assumptions — the model is too simple), variance (error from sensitivity to the training sample — the model is too complex), and irreducible noise. You trade bias against variance by tuning model complexity.
- High bias / underfitting: model too simple, misses real structure. High train error AND high test error.
- High variance / overfitting: model memorizes the training set. Low train error but high test error.
- The gap between train and validation error is your variance signal; a high floor on both is your bias signal.
The dartboardBias is throwing tightly grouped darts that all miss the bullseye (consistent but wrong). Variance is darts scattered all around it (sometimes right, but unstable). You want tight AND centered.
Say the decompositionNaming bias + variance + irreducible error and then diagnosing from the train-vs-validation gap is exactly what interviewers listen for — it shows you can debug a model, not just define terms.
◆ Lock it in
- Error = bias + variance + irreducible noise.
- Underfit = high bias (both errors high); overfit = high variance (big train/val gap).
- More complexity lowers bias but raises variance — tune to the sweet spot.
Feynman drill — say it out loudExplain, using the train and validation error, how you'd tell whether a model is underfitting or overfitting.
Step 1 rate your confidence · Step 2 pick your answer
A model scores 99% on training data but 71% on validation. What is happening?
Step 1 — how sure are you?