DevOps & SRE Interview Questions
The DevOps / SRE interview covers CI/CD, containers and Kubernetes, cloud, infrastructure as code, and observability/reliability (SLIs, SLOs, error budgets).
Interview questions & answers
Design a CI/CD pipeline for a containerized web service. Walk through the stages.
CI: on every push, build + run unit/integration tests, lint, and a security scan; keep the mainline green. Build once: produce an immutable image tagged by commit SHA and push it to a registry. Promote the same artifact through staging → prod; deploy with a canary or blue-green strategy and automated metric checks. Secure it: OIDC short-lived creds, signed artifacts, no plaintext secrets; roll back fast on failure.
Compare blue-green and canary deployments. When do you pick each?
Blue-green: two full environments, flip traffic to green — instant rollback, but double capacity briefly. Canary: route a small traffic slice to the new version, watch metrics, promote or abort — smallest blast radius. Pick blue-green for instant rollback when you can afford the extra fleet; canary when you want minimal user impact and have the metrics to auto-analyze.
Explain the difference between a container and a VM, and how you keep images small and safe.
Container shares the host kernel (namespaces + cgroups) — light and fast; a VM runs a full guest OS on virtualized hardware — heavier, stronger isolation. Small/safe images: multi-stage builds, minimal/distroless base, run as non-root, order layers for cache reuse. Scan images and pin/tag by commit SHA, not just 'latest'.
Walk from a Pod to external traffic. What does each Kubernetes object add?
Pod: smallest deployable unit, ephemeral, no stable IP. Deployment: keeps N replicas, handles rolling updates and rollbacks for stateless apps. Service: stable virtual IP/DNS that load-balances across current pods by label. Ingress: external HTTP(S) routing and TLS termination into Services.
A pod keeps restarting in a crash loop right after deploy. How do you debug it?
Check kubectl describe pod and events; look at logs (and previous container logs) for the crash reason. Common causes: failing liveness probe during slow startup (use a readiness/startup probe), bad config/secret, or hitting a memory limit (OOM-kill). Verify requests/limits, image tag, and env/config; roll back to the last good image if needed.
Explain the AWS shared responsibility model with examples.
AWS secures of the cloud: physical facilities, hardware, hypervisor, managed-service internals. You secure in the cloud: IAM policies, data encryption, OS patching on EC2, security groups and network config. The line shifts by service — more managed (Lambda, S3) means AWS owns more of the stack.
Name the Well-Architected pillars and give a design trade-off between two of them.
Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization, Sustainability. Trade-off example: multi-region for Reliability sharply increases Cost; tighter Security (encryption everywhere, inspection) can add latency hurting Performance. A senior answer names the trade-off chosen and why, rather than pretending an axis is free.
Why does Terraform need a state file, and how do you manage it on a team?
State maps declared config to real resource IDs so Terraform can compute the diff (create vs update vs destroy) and detect drift. Use remote state (S3 with native locking / Terraform Cloud; older setups pair S3 + DynamoDB) so the team shares one truth, with locking to prevent concurrent-apply corruption. State can contain secrets in plaintext — encrypt the backend, restrict access, never commit it to Git.
What are the three pillars of observability and when do you reach for each?
Metrics: numeric time series — cheap, for dashboards and alerting on rate/errors/latency. Logs: detailed structured events — for the 'what exactly happened' on a specific request. Traces: one request's path across services — to find where latency/errors occur in a distributed call. Alert on a metric; then use traces/logs to investigate the why. Emit all three via OpenTelemetry.
Explain SLIs, SLOs, and error budgets, and how a team uses them.
SLI = measured signal of user-facing health (success rate, p99 latency). SLO = target over a window (99.9% over 30 days). Error budget = 100% minus the SLO — the allowed unreliability (99.9% is about 43 min/month). Decision rule: budget left → ship features; budget spent → freeze risk, invest in reliability. It aligns dev and ops and kills the '100% uptime' argument.
Production is down. Walk me through your incident response and what happens after.
Declare an incident, assign an Incident Commander (coordinates, does not fix), open a comms channel and timeline. Mitigate first: roll back or fail over to restore service; diagnose root cause after. Optimize MTTR. After: write a blameless postmortem — timeline, impact, and systemic action items that fix the system, not blame a person.
How does serving an LLM in production differ from a normal web service?
It is GPU-bound and non-deterministic: VRAM usually binds before compute, so batching and KV-cache management drive throughput. Use an inference server (vLLM/SGLang/TGI/Triton); autoscale on GPU utilization / queue depth, and measure time-to-first-token and tokens/sec, not just RPS. Same DevOps discipline applies: version + canary models with evals, control cost via quantization, caching, and routing easy requests to smaller models.
'A production server is slow.' Debug it live — talk me through it.
Fixed first minute: uptime, dmesg -T | tail, top, free -h, df -h + iostat -xz, ss -s — narrating a hypothesis before each command. Interpret: high load + idle CPU → iowait; swap in use → memory pressure; OOM-killer lines in dmesg → something got killed. Service level: systemctl status and journalctl -u app --since; correlate with recent changes (deploy, config, cron). Frame with the USE method so no resource is skipped; state the fix and how you'd confirm recovery.
Services report intermittent DNS failures inside the Kubernetes cluster. Debug it.
Reproduce from a debug pod: nslookup service.namespace.svc and an external name — internal-only vs all-lookups narrows it immediately. Check CoreDNS: pods healthy? throttled or OOM-killed (resource limits)? errors in its logs? Know the classics: ndots:5 amplifying external lookups, conntrack table exhaustion on nodes, flaky upstream resolvers. Mitigate (node-local DNS cache, more CoreDNS replicas), then fix the root cause.
Design a 'golden path' so 15 product teams can ship new microservices without tickets.
A scaffold/template that generates a repo with CI pipeline, Dockerfile, k8s manifests, and observability pre-wired (metrics/logs/traces, SLO starter). Self-service via an internal developer platform / service catalog — teams provision without filing tickets. Guardrails, not gates: scanning, policy checks and sane defaults baked in; escape hatches documented. Measure success by voluntary adoption and time-to-first-deploy — an unadopted platform is a failure regardless of elegance.
Your cloud bill doubled in a quarter. Bring it down without hurting reliability.
Visibility first: tagging + cost allocation per team/service; find the top line items before touching anything. Quick wins: delete idle/orphaned resources (unattached volumes, old snapshots, forgotten environments); right-size instances. Match pricing to workload shape: spot for stateless/batch, commitments for the steady baseline. Kubernetes: over-provisioned requests are invisible waste — right-size them; add storage lifecycle policies; make cost a visible per-team metric so it stays down.