ML & Data Science Interviews · Feature Engineering & Data · Lesson 2 of 3
Encoding, scaling & missing data
Most raw data isn't model-ready: categoricals need numbers, numeric ranges need to be comparable, and gaps need a decision. How you handle each affects both accuracy and leakage.
Encoding categoricals
- One-hot: a binary column per category. Safe and lossless, but explodes dimensionality for high-cardinality features.
- Label/ordinal: integer per category — only when order is meaningful (low/med/high), else models read false ordering.
- Target (mean) encoding: replace a category with the mean target for that category. Powerful for high cardinality, but leaks easily — compute it out-of-fold / with smoothing.
Scaling & missing data
- Standardization (z-score) vs min-max normalization: needed for distance/gradient models (kNN, SVM, linear, neural nets); trees don't care.
- Missing data: impute (mean/median/mode, model-based), or add a 'was-missing' indicator when missingness itself is informative. Drop only if rare and random.
Does the model care about scale?Distance- and gradient-based models (kNN, SVM, linear/logistic, neural nets) need scaling. Tree-based models (RF, XGBoost) are invariant to monotonic feature scaling — a quick way to sound like you know why.
◆ Lock it in
- One-hot for low cardinality; target encoding for high cardinality (compute out-of-fold to avoid leakage).
- Scale features for distance/gradient models; trees don't need it.
- Impute missing values; add a missingness indicator when the gap is informative.
Feynman drill — say it out loudExplain when target encoding beats one-hot, and why it can leak if done carelessly.
Step 1 rate your confidence · Step 2 pick your answer
You have a 'zip_code' feature with 10,000 distinct values. Which encoding is most appropriate, and what's the catch?
Step 1 — how sure are you?