Machine Learning Interview Questions & Answers
Classic machine learning and data-science interview questions — the fundamentals, algorithms, metrics, statistics and ML system design.
Interview questions & answers
Explain the bias-variance tradeoff and how you'd diagnose which one is hurting your model.
Generalization error = bias + variance + irreducible noise. Bias = too-simple assumptions; variance = over-sensitivity to the training sample. Diagnose from errors: high train AND high validation error → high bias (underfit). Low train, high validation (big gap) → high variance (overfit). Fix high bias: more complex model / features. Fix high variance: regularization, more data, simpler model, early stopping.
L1 vs L2 regularization — mechanism and when you'd choose each.
Both add a penalty on weight size to the loss, trading a little bias for less variance. L1 (Lasso) penalizes |w|; its cornered constraint region zeroes weights → sparsity / feature selection. L2 (Ridge) penalizes w^2; shrinks all weights smoothly, good when many features each contribute a bit. Choose L1 to select from many features; L2 (or Elastic Net) for correlated features / stability.
When would you use gradient boosting over a random forest, and vice versa?
Boosting (XGBoost/LightGBM): usually higher accuracy on tabular data; sequential trees cut bias. Use when you can tune and want top performance. Random forest: robust, low-tuning, parallel, hard to overfit; great baseline and when you want stability over peak accuracy. Both need no scaling and capture interactions. Boosting is more sensitive to hyperparameters and overfitting if unregularized.
Walk me through PCA and when it's useful.
Unsupervised dimensionality reduction: find orthogonal directions (principal components) of maximum variance and project onto the top k. Useful to compress correlated features, denoise, speed up training, and visualize in 2D/3D. Caveats: components are linear combinations (less interpretable), it's variance-based so scale features first, and it's not for supervised feature selection.
Accuracy is 98% but the model is useless. What happened and what do you report instead?
Almost certainly class imbalance: predicting the majority class gets high accuracy while missing the minority entirely. Report from the confusion matrix: precision, recall, F1, and PR-AUC for the positive class. Fix the modeling: resampling / class weights, tune the threshold, and keep the test set at the true distribution.
ROC-AUC vs PR-AUC — when does the distinction matter?
ROC-AUC = ranking quality (P(positive ranked above negative)) across thresholds; insensitive to the threshold but can flatter models under imbalance. PR-AUC focuses on the positive class (precision vs recall) and is far more informative when positives are rare. Rule of thumb: heavy imbalance → prefer PR-AUC; balanced classes → ROC-AUC is fine.
What is data leakage? Give examples and how you prevent it.
Leakage = information not available at prediction time entering training → inflated offline scores that collapse in production. Examples: target leakage (feature derived from the label), preprocessing contamination (fit scaler/encoder on all data before split), temporal leakage (using the future). Prevent: fit all preprocessing on the training fold only (pipelines), split by time when relevant, and audit features that dominate importance.
How do you encode a high-cardinality categorical feature?
One-hot explodes dimensionality, so prefer target/mean encoding, frequency encoding, or learned embeddings. Target encoding must be computed out-of-fold (or with smoothing/leave-one-out) to avoid label leakage. Consider grouping rare categories into an 'other' bucket to reduce noise.
Explain p-values and confidence intervals, and one common misinterpretation of each.
p-value = P(data at least this extreme | null true); compare to alpha to reject or not. Misread: it is not P(null is true). Confidence interval = a range that contains the true parameter X% of the time under repeated sampling. Misread: it is not 'a 95% probability the parameter is in this interval'. Also stress: statistical significance is not practical significance — report effect size.
Design an A/B test for a new feature and list the main pitfalls.
Define the metric and hypothesis, compute sample size / power for the minimum effect worth detecting, fix duration up front. Randomize users, verify balance (A/A test), pick a significance level. Pitfalls: peeking (early stopping inflates false positives), multiple comparisons, underpowering, novelty effects, and interference/network effects. Report the effect size and confidence interval, then validate the online metric moved.
Design a recommendation system for an e-commerce homepage.
Frame: goal (engagement/revenue) → ranking problem; constraints (latency, catalog scale). Metrics: offline NDCG/precision@k, online CTR/conversion. Data/labels: implicit feedback (clicks, purchases, dwell); handle position bias and the cold-start problem. Approach: baseline (popularity / collaborative filtering) → candidate generation + ranking (two-stage) with user/item/context features from a feature store. Serve & monitor: low-latency ranking, A/B test the change, watch drift and feedback loops; retrain on a schedule.
Your model looks great offline but underperforms in production. Diagnose.
Check training/serving skew: features computed differently in the two paths (top cause). Share a feature pipeline / feature store. Check data/concept drift: has the input distribution or the label relationship changed since training? Check leakage inflating the offline number, and metric mismatch (offline proxy not tied to the online business metric). Instrument monitoring on inputs, predictions, and outcomes; validate every change with an A/B test.
Explain gradient descent and the role of the learning rate. What goes wrong at each extreme?
Define a loss, compute its gradient w.r.t. the parameters, step downhill: w = w - lr * grad; repeat over mini-batches (SGD). Too high: overshoot — loss oscillates or explodes. Too low: training crawls and can stall in a poor region. Momentum/Adam smooth and scale the steps per parameter — Adam is the robust default; learning rate is still the first knob to tune. Mini-batch noise is a feature: cheaper steps and it helps escape bad regions.
What does self-attention do, and why did transformers replace RNNs for sequence tasks?
Each token scores its relevance against every other token and takes a weighted sum of their representations → context-aware embeddings ('bank' by 'river' ≠ 'bank' by 'loan'). RNNs pass one hidden state step by step: sequential (slow to train) and long-range information decays through the bottleneck. Attention connects any two positions directly and computes all positions in parallel — better long-range modeling and vastly better hardware utilization. Cost: attention is O(n^2) in sequence length — the reason context windows are a constraint.
Prompting vs RAG vs fine-tuning — how do you choose, and how do you evaluate the result?
Escalate: prompting first (cheapest, instant iteration) → RAG when answers need fresh, private, or citable knowledge → fine-tuning when you need consistent style/format/domain behavior. Key distinction: RAG changes the knowledge available at inference; fine-tuning changes the behavior in the weights — it is unreliable for adding facts. Evaluate with a golden set + rubric, pairwise comparisons, and LLM-as-judge validated against human labels; for RAG also measure retrieval quality and groundedness. Validate online with an A/B test — same offline-proxy/online-truth discipline as classic ML.
Build and validate a model to forecast weekly demand. What changes versus a standard regression problem?
Validation changes first: no shuffling — use a time-based split or rolling-origin (expanding-window) evaluation so validation is always in the future of training. Baselines: naive (last value) and seasonal naive (same week last year/cycle) — a model that can't beat these isn't working. Features: lags, rolling means, and calendar effects (holidays, seasonality); gradient boosting on lag features and classical methods (exponential smoothing / ARIMA) are both defensible — justify by data size and seasonality. Metrics: MAE/RMSE in units; be careful with MAPE near zero-demand weeks. Watch for temporal leakage — any feature computed with future information.