AI Agents Engineering · Memory, Eval, Safety & Production · Lesson 2 of 3
Evaluation, tracing & observability
Agents fail in the middle (bad retrieval, wrong tool, timeout, loop) — often before the final answer. So you must evaluate the trajectory, not just the output, and observe it in production.
- Offline evals: 20–50 golden tasks harvested from real failures with reference solutions; measure task success, steps taken, cost, latency, and tool-error rate. Run in CI.
- Trajectory vs outcome: score the path (sane tool choices/order) and the outcome — the final environment state (the refund actually processed), not the agent's claim. Beware 'corrupt success': right end state via an unsafe path.
- pass@k vs pass^k: pass@k (any of k runs succeeds) measures capability; pass^k (ALL k succeed) measures the reliability a customer feels — 90% per-run is only ~35% over 10 runs.
- LLM-as-judge: binary pass/fail per failure mode, validated against human labels (TPR/TNR) — and don't assert exact tool-call sequences; that's a named anti-pattern that punishes valid creative paths.
- Observability: trace every step — inputs, tool calls, latency, tokens — via OpenTelemetry GenAI conventions, with LangSmith / Langfuse / Braintrust as swappable backends.
You cannot improve what you cannot seeThe single highest-leverage production investment is tracing. When an agent misbehaves, a full trace of thoughts, tool calls, and results turns a mystery into a two-minute diagnosis.
◆ Lock it in
- Evaluate the trajectory (tool calls, steps, cost, latency), not just the final output.
- Combine golden-task evals in CI + LLM-as-judge for open-ended outputs.
- Trace everything (LangSmith / Langfuse / Braintrust) — agents fail before the answer.
Feynman drill — say it out loudWhy is evaluating an agent harder than evaluating a single LLM call, and what does a trace give you?
Step 1 rate your confidence · Step 2 pick your answer
Why must agent evaluation look at the trajectory, not just the final answer?
Step 1 — how sure are you?