ML & Data Science Interviews · Deep Learning Fundamentals · Lesson 2 of 2
Embeddings, attention & LLM-era questions
Generalist ML/DS breadth rounds now routinely include an LLM-era question alongside the classics. You do not need to derive a transformer — you need crisp one-paragraph explanations and one decision framework.
- Embeddings: dense vectors where semantic similarity becomes geometric closeness (cosine similarity). They power semantic search, recommendations, clustering, and retrieval.
- Attention: each token scores its relevance against every other token and takes a weighted sum of their representations — so 'bank' near 'river' and 'bank' near 'loan' end up with different, context-aware vectors.
- Transformer: embeddings plus stacked attention + feed-forward layers, trained on next-token prediction at scale. It displaced RNNs because attention sees long-range context directly and trains in parallel across positions instead of one step at a time.
Prompting vs RAG vs fine-tuning
- Prompting first: cheapest, no training, instant iteration. Exhaust it before anything heavier.
- RAG (retrieval-augmented generation): retrieve relevant documents at inference and ground the answer in them. Choose it for knowledge that is fresh, private, or must be cited.
- Fine-tuning: further training that changes weights. Choose it for consistent style, format, or domain behavior — it is unreliable as a way to add facts.
- They compose: a fine-tuned model with RAG and a good prompt is a normal production answer, not a contradiction.
Fine-tuning is not a knowledge uploadThe classic wrong answer: 'our docs change weekly, so we fine-tune weekly.' Changing facts is a retrieval problem (RAG); fine-tuning is for changing behavior. Getting this backwards is the fastest way to fail the question.
Evaluating generative output — the follow-up you should expect: there is no single ground-truth label, so use a golden set scored against a rubric, pairwise comparisons between versions, LLM-as-judge validated against human labels on a sample, task-specific checks (for RAG: groundedness — is the answer supported by the retrieved text — and retrieval quality), and an online A/B test for the business metric. Note the symmetry: offline proxy + online validation, exactly like classic ML.
The framework beats the buzzwordsAnswer LLM questions with the same shape as classic ML design: objective → simplest approach first (prompting) → escalate with justification (RAG, then fine-tuning) → evaluation plan. Structure is what is being graded; provider names are not.
◆ Lock it in
- Embeddings turn meaning into geometry; attention builds context-aware representations; transformers train on next-token prediction, in parallel.
- Escalate prompting → RAG → fine-tuning: RAG for fresh/citable knowledge, fine-tuning for consistent behavior.
- Evaluate generative output with golden sets, pairwise comparison, validated LLM-as-judge, and online A/B — proxy offline, validate online.
Feynman drill — say it out loudExplain to a PM when you would use RAG versus fine-tuning for a company chatbot, and how you would know it is working.
Step 1 rate your confidence · Step 2 pick your answer
Your assistant must answer questions over internal documents that change weekly, with citations. The right primary approach is:
Step 1 — how sure are you?