Product Manager Interview (+ AI PM) · Metrics & Analytics · Lesson 2 of 3
Diagnosing a metric drop
'Metric X dropped Y% — investigate' is a staple analytical round. It grades structured hypothesis generation, not a lucky guess. Run a repeatable diagnostic before proposing any cause.
- Clarify + quantify: which metric exactly, how much, over what window, sudden or gradual? Is it a real drop or a reporting/logging bug?
- Check for seasonality / external events: holiday, weekend, a competitor launch, a news event.
- Segment the drop: by platform (iOS/Android/web), geography, new vs existing users, app version, channel. A drop isolated to one segment points at the cause.
- Internal changes: did we ship a release, change pricing, alter onboarding, or break a payment flow? Correlate timing.
- Form hypotheses, then validate: state the most likely cause and the data you'd pull to confirm it.
Internal vs external vs measurementBucket causes into three: measurement (broken tracking), internal (we shipped/changed something), external (seasonality, competitor, macro). Covering all three shows rigor and stops you fixating on one story.
The strongest tell is segmentation: 'is the drop across the board or concentrated?' If Android-only after a release, you have your prime suspect in one move.
◆ Lock it in
- First rule out a measurement/logging artifact, then investigate real causes.
- Bucket causes into measurement, internal, external; check all three.
- Segment the drop (platform, geo, new vs existing, version) to localize the cause.
Feynman drill — say it out loudOutline how you'd investigate 'daily active users dropped 15% overnight' before naming a cause.
Step 1 rate your confidence · Step 2 pick your answer
What should you rule out FIRST when a metric suddenly drops?
Step 1 — how sure are you?