AI System Design · The Framework · Lesson 1 of 3
Clarify before you architect
The AI system design round tests judgment under ambiguity, not trivia. The single biggest differentiator: candidates who clarify requirements first vs. those who start drawing boxes. Spend the first 3-5 minutes turning a vague prompt ("design an AI assistant for our docs") into a concrete, scoped problem.
The clarifying checklist
- Users & scale: who, how many, QPS at peak, concurrent sessions?
- Task type: Q&A, summarization, extraction, classification, agentic actions, generation?
- Data: how much, what modality, how fresh, where does it live, who can see what?
- Latency budget: interactive (<2s to first token) vs. batch (minutes ok)?
- Quality bar: what does 'good enough' mean, and what's the cost of a wrong answer?
- Constraints: cost ceiling, on-prem/VPC, compliance (HIPAA/GDPR/SOC2), model provider limits.
Say this out loud"Before I design anything, let me pin down scale, latency, the quality bar, and data/compliance constraints — those decide the whole architecture." This one sentence reads as senior and buys you thinking time.
Write the answers on the board as explicit assumptions. If the interviewer doesn't specify, propose a number ("let's assume 100 QPS peak, 50M documents") — a stated assumption you can design against beats waiting for permission.
◆ Lock it in
- Open with clarifying questions, not boxes — it's the top signal of seniority.
- Nail scale, task type, data, latency, quality bar, constraints.
- State explicit assumptions with concrete numbers and design against them.
Feynman drill — say it out loudYou're told 'design an AI assistant for our support docs.' List the six things you'd clarify before drawing anything.
Step 1 rate your confidence · Step 2 pick your answer
In an AI system design interview, what should you do in the first five minutes?
Step 1 — how sure are you?