Take-homes & work trials: evals first, then the video

Most AI-native loops now gate on a build artifact: OpenAI's ~5-hour build-on-our-APIs with a recorded video walkthrough; Anthropic's 3–4-hour app from a fictional customer brief; Sierra/LangChain-style build-an-agent-for-a-fictional-client; and paid multi-hour work trials inside the company's real codebase. Typical calibration: 2–4 hours of actual work inside a multi-day window — scope ruthlessly.

What graders actually score

• It runs, end to end, on their machine, first try. A modest working thing beats an ambitious broken one.
• An eval harness — even 15–30 labeled cases with a pass/fail script. Hiring managers say it plainly: the red flag is a candidate who doesn't start with evals.
• Production hygiene: error handling, retries/backoff, logging, graceful degradation, no hardcoded keys.
• Judgment in writing: a short README with tradeoffs, cost/latency notes, and 'what I'd do with another week.'
• Handled ambiguity: the brief is deliberately underspecified — state your assumptions the way you would to a customer.

The video walkthrough (a customer-demo proxy)

1. Demo the working thing first (60–90 seconds).
2. Architecture in one diagram — why this shape.
3. One tradeoff you made consciously, and what you gave up.
4. Your evals: what you measured and what it showed.
5. What you'd do next. Keep it ≤5 minutes, energy up — they're imagining you in front of their customer.