ML & Data Science Interviews · Core Algorithms · Lesson 3 of 3
SVM, kNN, k-means & PCA
The rest of the algorithm zoo interviewers love — two more supervised learners, plus the two unsupervised staples. Know the one-line intuition and when each shines.
- SVM (supervised): finds the hyperplane with the maximum margin between classes; the kernel trick handles non-linear boundaries. Strong in high dimensions / smaller datasets; needs scaling.
- kNN (supervised, lazy): classify a point by the majority vote of its k nearest neighbors. No training, but slow at prediction and sensitive to scaling and the curse of dimensionality.
- k-means (unsupervised): partition points into k clusters by iteratively assigning to the nearest centroid and recomputing centroids. Fast; assumes roughly spherical, similar-size clusters; you must pick k (elbow / silhouette). Alternatives to name: DBSCAN (arbitrary shapes, outliers, no k) and hierarchical (dendrogram, small data).
- PCA (unsupervised): dimensionality reduction — project onto the orthogonal directions of maximum variance. Compresses correlated features, denoises, and aids visualization; components are combinations, so interpretability drops.
Supervised vs unsupervisedA common trap: mixing these up. SVM and kNN need labels (supervised). k-means and PCA do not (unsupervised). And k-means (clustering) is not kNN (classification) — different tasks despite the 'k'.
◆ Lock it in
- SVM: max-margin boundary + kernels; kNN: vote of nearest neighbors, lazy, scale-sensitive.
- k-means: partition into k clusters via centroids (unsupervised); pick k with elbow/silhouette.
- PCA: reduce dimensions along directions of max variance (unsupervised).
Feynman drill — say it out loudSort SVM, kNN, k-means and PCA into supervised vs unsupervised, and say what each is for in one line.
Step 1 rate your confidence · Step 2 pick your answer
Which pair are both UNSUPERVISED methods?
Step 1 — how sure are you?