DevOps, SRE & Cloud Interview · Observability & SRE · Lesson 1 of 3
The three pillars: metrics, logs, traces
Monitoring tells you whether the system is healthy against known questions; observability is the property that lets you ask new questions about why — to debug states you did not predict. Three signal types power it.
- Metrics: numeric time series (request rate, error rate, latency, CPU). Cheap to store, great for dashboards and alerts; low cardinality.
- Logs: timestamped, often structured event records. Rich detail for a specific event; expensive at volume. Prefer structured (JSON) logs.
- Traces: the path of one request across services, as a tree of timed spans. They reveal where latency and errors happen in a distributed call.
The RED and USE methodsTwo useful defaults: RED for request-driven services — Rate, Errors, Duration. USE for resources — Utilization, Saturation, Errors. Instrument these first and most dashboards write themselves.
Logs are not a metrics substituteGrepping logs to compute an error rate is slow and expensive. Emit a metric for the rate (alert on it), and reach for logs/traces to investigate the why once a metric fires. Use OpenTelemetry to emit all three consistently.
◆ Lock it in
- Metrics (numbers, cheap, alerting), logs (detailed events), traces (request path across services).
- Monitoring = known questions; observability = ask new 'why' questions about unforeseen states.
- Instrument RED (services) and USE (resources) first; standardize on OpenTelemetry.
Feynman drill — say it out loudExplain when you would reach for a trace versus a metric versus a log while debugging a slow endpoint.
Step 1 rate your confidence · Step 2 pick your answer
Which signal best reveals WHERE latency is spent across a request that spans several microservices?
Step 1 — how sure are you?