ML & Data Science Interviews · Feature Engineering & Data · Lesson 1 of 3
Data leakage: the silent killer
Data leakage is when information unavailable at prediction time sneaks into training, inflating offline scores that then collapse in production. It is the single most common reason a 'great' model fails live — and a favorite interview probe.
- Target leakage: a feature is a proxy for, or computed from, the label (e.g. 'account_closed_date' when predicting churn).
- Train/test contamination: fitting scalers, encoders, or imputers on the full dataset before splitting — the test set's statistics leak into training.
- Temporal leakage: using future information to predict the past; always split by time for time-dependent problems.
- The tell: suspiciously high offline metrics and a single feature dominating importance.
Fit preprocessing inside the foldFit scalers/encoders/imputers on the training fold only, then transform validation/test. Wrapping preprocessing in a pipeline that's fit per-fold is the clean, leak-proof way — do this and cross-validation stays honest.
◆ Lock it in
- Leakage = info unavailable at prediction time entering training → inflated, fake scores.
- Watch for target leakage, preprocessing contamination, and temporal leakage.
- Fit all preprocessing on train only; split by time when time matters.
Feynman drill — say it out loudGive a concrete example of target leakage and explain how you'd catch it before shipping.
Step 1 rate your confidence · Step 2 pick your answer
Which of these is a textbook example of data leakage?
Step 1 — how sure are you?