- 5-10 years experience. - Roles: Site Reliability Engineer, Senior SRE, Staff SRE. - 70 checklist questions, 30 real interview Q&A, 10 live scenarios, 15 behavioral questions. - Companies: Razorpay, Zerodha, Swiggy, Hotstar, PhonePe.
You are 5+ years into your career and interviewing for a Senior or Staff SRE role. The bar shifts from "can you operate a system" to "can you design, defend, and own the reliability of a system under real business constraints." Interviewers expect you to state SLOs before drawing diagrams, quantify blast radius before proposing a fix, and defend trade-offs under pushback. Tier 1 is your readiness gate. Tier 2 is 30 fully-answered senior-level questions. Tier 3 is 10 live production scenarios with no single correct answer. The Behavioral Round is 15 STAR-format questions on incident ownership, technical leadership, and cross-team conflict. ---
No answers given. If you cannot answer these cold, study first. ### Linux and Systems * What is the difference between TIME_WAIT and CLOSE_WAIT, and which side of a connection enters each? * What does a process's oom_score represent and how is it calculated? * What is the difference between a process's soft and hard ulimit? * How does the CFS scheduler decide which process runs next? * What is CPU steal time and when does it matter? * What is the difference between cgroups v1 and v2? * What does `/proc/<pid>/status` show that `ps aux` does not? * What is a zombie process and how do you actually clear one? * What is the SYN queue versus the accept queue? * What does iostat's `util` column actually measure? * What is head-of-line blocking and how does HTTP/2 partially fix it? * What is deadline propagation and why does a per-hop timeout not achieve the same thing? ### Kubernetes and Containers * What is the difference between a liveness and a readiness probe? * What happens inside the kernel when `resources.limits.memory` is set on a pod? * Why can a pod be OOMKilled while the host has free RAM? * What is the difference between PodFailure and PodKill in a chaos testing context? * What causes a PVC to be stuck Pending after a node failure? * What does kube-proxy actually do, and how does its behavior differ between iptables and IPVS mode? * Why does a gRPC service need a dedicated probe type instead of a TCP probe? ### Distributed Systems * State the CAP theorem in one sentence without naming a specific database. * What is split-brain and how does Raft's majority-vote rule prevent it? * Why does a 5-node cluster tolerate the loss of only 2 nodes, not more? * What is the difference between full jitter and equal jitter in retry backoff? * What is a bulkhead pattern and what specific failure does it prevent? * Why is "timeout" not the same as "confirmed failure" for a retried operation? * What does an idempotency key actually protect against? ### Databases * What does PostgreSQL's WAL underlie besides crash recovery? * What is transaction ID wraparound and why is it a hard stop, not a warning? * What is the difference between PgBouncer's session, transaction, and statement pool modes? * Why can a database show near-zero CPU while every request times out? * What is the difference between RDB and AOF persistence in Redis? * When should you choose Redis Cluster over Redis Sentinel? * What two things does point-in-time recovery actually require? ### SRE Practice * What are the six defining characteristics of toil? * Why should a team's toil target never be set to exactly 0%? * What is the toil-elimination decision tree, in order? * What is the difference between an SLI, an SLO, and an error budget? * Why should DORA metric improvement be described as something that "can" happen, not something that automatically does? * What is the USE Method and what three things does it check per resource? * What is a blast radius and why should a first chaos experiment minimize it? ### Networking * What does EADDRNOTAVAIL on an outbound connection almost always indicate? * Why does lowering a DNS TTL before a migration not by itself speed up propagation? * What is a DNS cache stampede? * What does a certificate's notAfter expiry cause, versus a gradual warning? * What is the difference between a SYN queue overflow and an accept queue overflow? * What does `net.core.somaxconn` control, and why can raising it alone do nothing? ### Observability and Incident Response * What are the four Golden Signals? * Why is "the SLO held" not sufficient to call a chaos experiment fully successful? * What is the difference between Detection, Resilience, and Recovery in a chaos experiment? * Why should incident response usually not be automatically classified as toil? * What should a toil-reduction report include beyond the percentage reduction alone? ---
### System Reliability and SLOs **Q1. Design an SLO for a checkout API that currently has none. Walk through your reasoning.** Start with the SLI, not the target. Pick something the user actually experiences - percentage of checkout requests completing under 800ms with a non-error status - rather than a raw uptime metric nobody can act on. Set the target based on real business tolerance, not a round number. State it explicitly: "99.9%, not 99.99%, because the cost of chasing the next nine is exponential and the business can tolerate a small, bounded rate of failed checkouts more than it can tolerate the engineering cost of near-perfect availability." Define the error budget in concrete terms - roughly 43 minutes of full-downtime-equivalent per 28-day window at 99.9% - and define what burning it triggers: a deploy freeze on checkout-adjacent services if 50% of the budget burns in any 7-day window. A strong answer closes by naming the trend, not just the threshold: alert on the rate of change of budget burn, since a lag or error rate that is small but climbing is a much earlier signal than waiting for a fixed line to be crossed. **Q2. Why should you never set a team's toil target to 0%?** Any on-call rotation creates a structural floor of unavoidable interrupt-driven work, roughly the number of on-call weeks divided by the rotation size. A six-person rotation running one primary and one secondary week per cycle already has roughly a 33% floor before any additional toil is counted. Setting a 0% target ignores this floor entirely and sets the team up to chase a number that was never reachable, which demoralizes rather than motivates. **Q3. A chaos experiment kills a pod, the SLO holds, but no alert fired and nobody noticed. Was the experiment successful?** No - or at least, not fully. This is exactly why chaos experiments validate three separate categories: Resilience (did the SLO hold), Detection (did monitoring notice), and Recovery (did the system self-heal or need a runbook). The system was resilient here, but the silent failure of detection is the more valuable finding - a real-world version of this failure, possibly worse, could have gone completely unnoticed. The correct response is to treat this as a genuine finding and fix the missing alert, not close the experiment as a clean pass. ### Linux and Kubernetes Depth **Q4. A service is "active (running)" per systemctl, CPU and memory look fine, but it's returning 503s. What's your first hypothesis and how do you confirm it?** File descriptor exhaustion in an upstream dependency, not the service itself. Confirm with `ls /proc/<pid>/fd | wc -l` against `/proc/<pid>/limits`. When a process hits its FD ceiling, every new socket call fails including accept() on new connections, while the process stays alive and every basic health signal stays green - this is exactly why `systemctl status` and `top` can both report healthy while the service is functionally down. **Q5. Explain why a pod can be OOMKilled while `free -h` on the host shows gigabytes free.** Kubernetes writes the pod's memory limit into its own cgroup (`memory.max` in v2, `memory.limit_in_bytes` in v1). When usage crosses that number, the kernel's OOM killer fires *inside that specific cgroup*, killing the largest process in it, entirely independent of host-wide memory state. The fix is raising the pod's own memory limit in its resource spec, never adding RAM to the node or tuning host-level swappiness - neither touches the actual enforcement boundary. **Q6. Round robin load balancing sends a new request to a pod already processing a 10-second report query. What's the failure mode, and does switching to least-connections fully fix it?** Round robin is blind to current load - it sends work in strict rotation regardless of how busy a backend already is. Least-connections improves this but is not a full fix: "fewest connections" is not the same as "least busy." A pod with 2 connections each stuck in a 30-second query can look less loaded than a pod with 50 connections serving 5ms cache reads, and least-connections can therefore keep routing more traffic toward the pod that is actually struggling. ### Distributed Systems and Consensus **Q7. A 5-node etcd cluster splits into groups of 2, 2, and 1 during a network partition. What happens, and why?** No group has a majority - a majority of 5 requires 3. All three groups stop accepting writes, even though every individual node is technically still alive. This is Raft correctly protecting against split-brain: quorum loss is not only about nodes crashing, a pure network partition can strand every node "up" while leaving no side with enough votes to safely elect a leader. **Q8. Why is "the request timed out" not the same as "the request failed," and why does this distinction matter for retry safety?** A timeout means no response was received in time - it says nothing about whether the operation completed server-side. The original write may have succeeded and only the response was lost. Retrying a non-idempotent operation under this assumption, such as a payment charge, can produce a duplicate side effect. This is why every retry pattern in production assumes idempotency as a precondition, not an optional nicety. ### Databases **Q9. A team disables autovacuum "temporarily" during a bulk import to improve write throughput. What risk does this introduce, and how would you monitor for it?** Transaction ID wraparound. PostgreSQL identifies every transaction with a 32-bit ID that can wrap after roughly 2 billion transactions; autovacuum is what keeps the oldest unfrozen transaction ID (`datfrozenxid`) bounded. Left disabled long enough, the database eventually refuses all new writes as a last-resort protection against silent data corruption - not a bug, but the database protecting itself. Monitor with `age(datfrozenxid)` per database, alerting well before the ~200 million warning threshold, since the real failure already happened when autovacuum was disabled, not when the hard stop triggers. **Q10. Every dashboard - database CPU, memory, disk - looks calm, but the application is timing out on every request to the database. What do you check that isn't on those dashboards?** Connection pool utilization, specifically PgBouncer or equivalent sitting in front of PostgreSQL. A slow downstream call can hold connections open longer than usual, filling the pool's fixed slot count while PostgreSQL itself sits idle with plenty of spare capacity it was simply never asked to use. This is one of the most commonly lost hours in real incidents, precisely because every standard database dashboard looks completely normal. ### Chaos Engineering and Toil **Q11. Why should a first chaos experiment always target the smallest blast radius in the lowest-risk environment?** To validate the tooling, the rollback process, and the team's understanding of the steady-state hypothesis before any real user impact is possible. Progression should move from staging to production and from a single pod to a wider blast radius - never both changes at once - so that a mistake in either the experiment design or the tooling itself is caught cheaply rather than during a production-wide test. **Q12. A team's staging chaos automation has run cleanly for weeks with tested abort conditions, but they've never run a Game Day. Should they move straight to continuous production chaos?** No. Staging automation proves the system-side tooling works; a Game Day proves the human-side incident response process works - whether the incident commander structure, runbooks, and communication cadence actually function under a live scenario. Skipping that stage removes the operational muscle needed to safely run automated production chaos, risking real incidents instead of controlled learning. ### Networking **Q13. A Node.js service opens a fresh outbound connection per request. Under load, new outbound calls start failing with EADDRNOTAVAIL. Diagnose it.** Ephemeral port exhaustion from connections stuck in TIME_WAIT. Each short-lived connection consumes a unique local source port, which sits unusable for roughly 60 seconds after close. At a high enough request rate, the ~28,000-port default range fills faster than TIME_WAIT drains it, leaving the kernel with no local port to assign for a new outbound connection. The durable fix is connection pooling at the application layer, not just widening the ephemeral port range or enabling `tcp_tw_reuse`, both of which only buy headroom rather than removing the cause. **Q14. CoreDNS pods are near their CPU limit during a traffic surge, and application latency spikes even though every downstream service reports healthy. Explain the connection.** A DNS lookup taking seconds instead of milliseconds under CoreDNS CPU throttling looks, from the calling application's point of view, identical to a slow downstream dependency - because the delay happens before the request even reaches any actual downstream service. Since every real downstream reports healthy, the bottleneck has to be introduced earlier in the path, at name resolution, which is exactly what CoreDNS CPU pressure produces. **Q15. Why can raising `net.core.somaxconn` alone fail to fix connection timeouts under load?** The kernel uses the smaller of two values: the application's own `listen()` backlog argument and the kernel-level `somaxconn` setting. If the application framework still passes a low backlog (many default to 128), raising only the sysctl value changes nothing, because the smaller application-level number remains the binding constraint on accept queue depth. --- *(Q16-Q30 continue this same depth across CI/CD reliability, capacity planning with Little's Law, incident command, cross-zone PVC scheduling, Redis eviction policy design, replication lag as an SLI, and multi-region failover trade-offs - each following the same pattern of a concrete production scenario, the underlying mechanism, and the correct fix.)* ---
**Q31. It's 3 AM. API pods are healthy, CPU is calm, memory is fine. Every request still times out. The database answers `SELECT 1` in milliseconds. Walk through your first ten minutes.** Do not trust the surface-level health signals - a database that answers a trivial query fast can still be functionally unreachable for real traffic. Check connection pool utilization first, since pool exhaustion produces exactly this signature: the bottleneck sits in a layer no standard database dashboard reports on. In parallel, check replication lag if reads are being served from a replica, and check for a stuck autovacuum or a long-running transaction holding locks that block everything behind it. Communicate early that the database itself is not yet ruled in or out - premature certainty here wastes the most time. **Q32. You are asked to reduce a service's toil by automating a recurring pod-restart task. The team wants to skip straight to writing a self-healing operator. What do you push back with?** Walk the toil-elimination decision tree in order before reaching for automation: can the root cause be fixed (why is the pod crash-looping at all), can the request simply be rejected or reduced, is there a self-service option that removes the need for a human without adding a new automated system to maintain. Automation is deliberately the fourth or fifth option, not the first, because it introduces a new system that itself needs monitoring, guardrails, and maintenance - and if the actual root cause (say, a memory limit set too low) can simply be fixed, that is stronger leverage than automating around it. **Q33. A self-healing operator you inherited restarts every CrashLoopBackOff pod with a 100% success rate and zero escalations, ever. Is this a good sign?** No - it is a red flag that classification is too permissive. An operator restarting blindly without checking the termination reason can mask a genuine root cause, such as a bad image or a missing secret, behind a loop of "successful" but pointless restarts while user impact never actually goes away. A healthy operator escalates a small, nonzero fraction of cases to a human; zero escalations ever is a sign to audit the classification logic, not a metric to celebrate. **Q34. During a live incident, you see `iostat -x` showing 85% util and climbing await on one disk, while `top` shows only 20% iowait. A colleague says "iowait is low, so it's not the disk." How do you respond?** iowait only accumulates when the CPU has nothing else to run and is specifically waiting on I/O - if other processes keep the CPU busy, iowait can stay low even while the disk itself is genuinely saturated. `util` is the actual disk saturation signal, and combined with climbing `await`, 85% util is real evidence regardless of what iowait reports. Redirect the investigation to the disk, not away from it. **Q35. A pod cannot reach a Kubernetes Service, but `kubectl get endpoints` shows a healthy, populated list. What's your next step, and why not just restart the pod?** Confirm kube-proxy itself is healthy and has synced recently, then inspect the actual NAT rule or IPVS entry it wrote for that specific Service. A Service IP is virtual - nothing listens on it directly, so a stale or missing kernel-level rule means kube-proxy hasn't synced correctly even though the Kubernetes API object looks correct. Restarting the destination pod doesn't touch that rule, so the same failure would likely recur. **Q36. Your team is migrating a monolith with no containerization history onto Kubernetes. What's the single biggest risk you'd surface before writing a single Dockerfile?** Stateful local files and in-memory session state. Containers are ephemeral by design - if the monolith writes logs, uploads, or session data to local disk or process memory, all of that disappears on restart or when traffic is split across multiple replicas. Move logs to stdout, uploads to object storage, and sessions to a shared store like Redis before attempting the container-to-Kubernetes move, and decouple containerization from the Kubernetes migration itself rather than doing both at once. **Q37. A short-TTL DNS record for a payments database causes a synchronized burst of client failures every time it expires under peak load, even though the database never went down. Diagnose and fix.** A DNS cache stampede - many clients' cached entries expire in the same narrow window and hit the resolver simultaneously, and if the resolver has even a brief hiccup at that exact moment, all of them fail together in a pattern that looks identical to a backend outage. The fix is not raising database capacity, which does nothing for a DNS-layer event; it's addressing resolver capacity or staggering TTL expiry so lookups don't cluster into synchronized waves. **Q38. You need to permanently resolve TIME_WAIT-driven port exhaustion on an outbound-heavy service. A teammate suggests enabling `tcp_tw_recycle`. Why is that the wrong answer, and what is?** `tcp_tw_recycle` was removed from modern kernels because it breaks connections from clients behind NAT, silently rejecting legitimate users who share a public IP. Kernel-level mitigations like `tcp_tw_reuse` or a wider ephemeral port range buy headroom but don't stop the service from opening one connection per request in the first place. The actual fix is connection pooling at the application layer, removing the root cause instead of tolerating the symptom. **Q39. A pod is stuck Pending after a node failure, and the on-call engineer is checking resource requests and taints. What are they missing?** A PersistentVolume is frequently pinned to a specific availability zone because the underlying block storage physically lives there. If the pod is rescheduled to a node in a different zone, Kubernetes cannot attach a zone-A volume to a zone-B node, and the pod stays Pending for a reason that isn't obvious from pod events alone. Confirm the PVC's actual zone with `kubectl get pv <name> -o jsonpath='{.spec.nodeAffinity}'` before spending more time on generic scheduling checks. **Q40. Leadership asks for a report on a toil-reduction automation project that cut a task from 15 minutes to 30 seconds across 12 incidents a month - a 96.7% reduction. What do you include beyond the percentage?** Baseline toil, residual human review time, and the escalation rate together, not the percentage in isolation. A high reduction paired with a healthy, nonzero escalation rate is a genuine win; the same percentage with zero escalations ever is worth re-auditing rather than celebrating, since it can indicate the automation's classification logic is too permissive and is silently absorbing cases it shouldn't. ---
**Q41. Tell me about an incident you handled that went badly. What did you change afterward?** Use STAR and close on a durable, verifiable change - not "we were more careful after that." Example shape: a migration caused a 40-minute checkout outage from underestimated lock contention; you made the call to roll back rather than ride it out; afterward you wrote a pre-migration checklist requiring a lock-duration estimate on any migration touching a large table, used on 30+ migrations since with zero repeat incidents of that type. The Result section should tie back to something concrete and reused, not just "resolved." **Q42. Describe a time you disagreed with a technical decision a more senior engineer had already made.** State your disagreement once, clearly and with specific technical reasoning, then commit fully to executing the decision if it stands. Half-hearted execution after losing a disagreement is the outcome interviewers are screening against - it produces worse results than either fully agreeing or escalating the disagreement through a proper channel. **Q43. Tell me about a time you owned a system nobody else understood well.** Focus on how you built understanding safely - reading logs and configs first, documenting as you went, avoiding changes until you had a real mental model - rather than jumping straight into "improvements" on a system whose failure modes you didn't yet understand. **Q44. How do you decide what to automate versus what to leave manual?** Reference the decision tree: root-cause fix first, then rejection or batching of low-value requests, then self-service, then automation, then human-in-the-loop with measurement. Automation is not the default response to any repetitive task - it's what you reach for after cheaper and more permanent options are ruled out. **Q45. Tell me about a time you had to communicate a technical risk to a non-technical stakeholder.** Strong answers translate the risk into business terms (dollars, downtime minutes, customer impact) without dumbing down the underlying mechanism, and describe a concrete decision that resulted from that conversation - not just that you "explained it well." **Q46. Describe a postmortem you wrote that changed how the team operated.** The differentiator is a specific, adopted process change with evidence it stuck - a new alert, a new checklist item, a new guardrail added to an automation - not just "we discussed it and moved on." **Q47. Tell me about a time you pushed back on a proposed automation because it lacked safety guardrails.** A strong example: an operator proposal that would restart any CrashLoopBackOff pod regardless of cause, and your pushback to add termination-reason classification, a restart cap, an audit log, and explicit escalation for unrecognized cases before it touched production. **Q48. How do you prioritize reliability work against feature deadlines?** Use error budget status as the deciding signal rather than gut feeling: a service comfortably within its budget can tolerate some toil and rough edges for now, while a service actively burning budget should preempt unrelated feature work regardless of deadline pressure. **Q49. Tell me about a time you mentored a junior engineer through a production incident.** Focus on how you let them drive while you guided the triage order, rather than taking over - and on a specific piece of judgment they demonstrably carried into future incidents afterward. **Q50. Describe the most technically complex system you've owned end-to-end.** Cover the failure modes you designed for explicitly - not just the happy-path architecture - and name one design decision you'd defend again under direct pushback. **Q51. Tell me about a time you had to say no to a request from a stakeholder because it would have hurt reliability.** The strongest answers show you offering an alternative, not just a refusal - for example, rejecting an urgent unreviewed production change but offering a fast-tracked, still-reviewed path instead. **Q52. How do you handle being paged for something that turns out to be a false alarm, repeatedly?** Treat repeated false alarms as a signal the alert itself needs fixing, not something to just tolerate - reference tightening the alert's duration threshold or its underlying query rather than normalizing being paged for noise. **Q53. Tell me about a time your on-call rotation structure itself was part of the problem.** Reference the structural toil floor concept directly - recognizing that a too-small rotation was creating an unsustainable baseline of interrupt work regardless of how well any individual incident was handled, and what you did about the rotation size or coverage model itself. **Q54. Describe a time you had to make a judgment call under incomplete information during an active incident.** Be explicit about what you knew, what you didn't, and why you chose to act anyway (or chose to wait) given the blast radius - this is a direct test of whether you can reason under uncertainty rather than freeze. **Q55. Where do you want to be in three years as an SRE?** A senior-level answer names a specific area of technical depth to go deeper on (distributed consensus, capacity planning, chaos engineering maturity) and a specific form of broader organizational impact - owning an SLO program across multiple teams, or building toil-elimination tooling used org-wide - rather than a vague title progression. ---
| Level | Salary Band | |:---|:---| | L4 - Senior SRE | Rs 30L - Rs 55L | | L5 - Staff SRE | Rs 55L - Rs 90L | | L6+ - Principal SRE | Rs 90L - Rs 150L | These bands assume confident, example-backed answers across Tier 2 and the Scenario Round. Candidates who answer only conceptually, without naming the specific mechanism (cgroup boundary, Raft quorum, TIME_WAIT) land at the lower end of each band.
You are 5+ years into your career and interviewing for a Senior or Staff SRE role. The bar shifts from "can you operate ...
No answers given. If you cannot answer these cold, study first. Linux and Systems What is the difference between TIMEWAI...
System Reliability and SLOs Q1. Design an SLO for a checkout API that currently has none. Walk through your reasoning. S...
Q31. It's 3 AM. API pods are healthy, CPU is calm, memory is fine. Every request still times out. The database answers S...
Q41. Tell me about an incident you handled that went badly. What did you change afterward? Use STAR and close on a durab...
Level Salary Band L4 - Senior SRE Rs 30L - Rs 55L L5 - Staff SRE Rs 55L - Rs 90L L6+ - Principal SRE Rs 90L - Rs 150L Th...
Aligns directly with DevOps, Site Reliability (SRE), and Platform Engineering job descriptions.