Overview and What You Will Learn
Without probes, Kubernetes has no way to know whether your application is actually healthy — it only knows if the container process is running. A Node.js API that is running but stuck in an infinite loop, a Spring Boot service that started but cannot connect to its database, or a pod that is ready to receive traffic before its cache is warmed — all of these appear healthy to Kubernetes without probes. This lab walks you through configuring all three probe types to achieve genuinely zero-downtime rolling deployments.
By the end of this guide you will be able to:
- Configure liveness probes to detect and automatically restart deadlocked or hung containers
- Configure readiness probes to gate traffic until the application is genuinely ready to serve requests
- Configure startup probes to protect slow-starting applications from premature liveness kills
- Design probe endpoints in your application that accurately reflect health state
- Tune probe timing parameters to balance responsiveness against false-positive restarts
Why This Matters in Production
During a Swiggy deployment at peak dinner hour, a new API version rolled out across 20 pods. The new version had a subtle bug — it started successfully, passed the liveness probe, but the readiness probe was not configured. Kubernetes sent live traffic to pods that had not finished loading their restaurant menu cache — resulting in 8 seconds of 500 errors for every user whose request hit an unready pod before the cache warmed up.
A correctly configured readiness probe would have held all traffic on the old pods until every new pod's cache was fully loaded — zero errors, zero user impact. Probes are the single most impactful configuration for achieving true zero-downtime deployments.
Core Principles
The three probe types and their distinct purposes: STARTUP PROBE LIVENESS PROBE READINESS PROBE ───────────── ────────────── ─────────────── "Has the app finished "Is the app still "Is the app ready starting up?" alive and not hung?" to receive traffic?"
Runs ONCE at startup Runs continuously Runs continuously until it succeeds throughout pod life throughout pod life
On failure → On failure → On failure → kubelet waits and kubelet RESTARTS pod REMOVED from retries (backoff) the container Service endpoints (no restart)
Disables liveness Triggers after Independent of probe while running startup probe liveness probe succeeds
The probe execution order for a new pod: Pod starts │ ▼ startupProbe runs (liveness disabled during this phase) │ ├─► Fails → kubelet waits periodSeconds → retries │ (up to failureThreshold × periodSeconds total) │ └─► Succeeds → startup complete │ ▼ livenessProbe + readinessProbe both begin running in parallel │ │ ▼ ▼ Failure → restart Failure → removed from container Service endpoints
Detailed Step-by-Step Practical Lab
Step 1 — Understand Probe Mechanisms
Kubernetes supports three probe mechanisms — choose based on what your application exposes:
# Kubernetes makes an HTTP GET to the specified path and port# Success: HTTP status 200-399# Failure: HTTP status 400+ or connection refused or timeoutlivenessProbe: httpGet: path: /healthz # Your health check endpoint port: 8080 httpHeaders: - name: Custom-Header value: kubernetes-probe # Mechanism 2 — TCP Socket (for non-HTTP services like databases, message queues)# Kubernetes attempts to open a TCP connection to the port# Success: connection established# Failure: connection refused or timeoutlivenessProbe: tcpSocket: port: 5432 # PostgreSQL port — if TCP accepts, pod is alive # Mechanism 3 — Exec Command (run a command inside the container)# Success: command exits with code 0# Failure: command exits with non-zero code or times outlivenessProbe: exec: command: - pg_isready # PostgreSQL readiness check binary - -U - postgres - -d - zerodha_tradingStep 2 — Configure a Complete Probe Suite for a Node.js API
# deployment-api-probes.yaml — full probe configuration for Swiggy order APIapiVersion: apps/v1kind: Deploymentmetadata: name: order-api namespace: productionspec: replicas: 5 strategy: type: RollingUpdate rollingUpdate: maxSurge: 2 # Allow 2 extra pods during update maxUnavailable: 0 # Never take a pod down before a new one is ready # This is what makes rolling updates truly zero-downtime selector: matchLabels: app: order-api template: metadata: labels: app: order-api spec: containers: - name: order-api image: registry.swiggy.in/order-api:v6.2.1 ports: - containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" limits: cpu: "500m" memory: "512Mi" # STARTUP PROBE — protects slow-starting apps from liveness kills # Total startup budget: 30 attempts × 10s = 300s (5 minutes maximum) startupProbe: httpGet: path: /healthz port: 8080 failureThreshold: 30 # Allow up to 5 minutes for startup periodSeconds: 10 # Check every 10 seconds successThreshold: 1 # One success is enough # LIVENESS PROBE — detects hung or deadlocked application # Only runs AFTER startupProbe succeeds livenessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 0 # startupProbe handles the delay — set to 0 periodSeconds: 15 # Check every 15 seconds timeoutSeconds: 5 # Fail if no response within 5 seconds failureThreshold: 3 # Restart after 3 consecutive failures (45s) successThreshold: 1 # READINESS PROBE — gates traffic until app is truly ready readinessProbe: httpGet: path: /ready port: 8080 # Different endpoint from liveness initialDelaySeconds: 5 periodSeconds: 5 # Check every 5 seconds — faster than liveness timeoutSeconds: 3 failureThreshold: 3 # Remove from endpoints after 3 failures (15s) successThreshold: 2 # Require 2 consecutive successes before adding backRememberUse
maxUnavailable: 0combined withmaxSurge: 1or higher for zero-downtime deployments. This forces Kubernetes to bring up new pods and wait for them to pass readiness before terminating old ones. Without this, Kubernetes may terminate old pods while new ones are still starting.
Step 3 — Implement Correct Health Endpoints in Your Application
The probe endpoint must accurately reflect the application's actual health — not just return 200 unconditionally:
// health.js — Node.js health endpoints for Swiggy order APIconst express = require('express');const router = express.Router(); // LIVENESS endpoint — am I alive and not deadlocked?// Should only check things that would require a restart to fixrouter.get('/healthz', async (req, res) => { try { // Check if the event loop is responsive (not deadlocked) // Check if critical in-process state is valid const memUsage = process.memoryUsage(); const heapUsedPercent = memUsage.heapUsed / memUsage.heapTotal; if (heapUsedPercent > 0.95) { // Heap is 95%+ full — restart before OOMKill hits return res.status(503).json({ reason: 'heap_pressure', heapUsedPercent }); } res.status(200).json({ status: 'alive', uptime: process.uptime() }); } catch (err) { res.status(503).json({ status: 'unhealthy', error: err.message }); }}); // READINESS endpoint — am I ready to serve traffic?// Check all dependencies the application needs to functionrouter.get('/ready', async (req, res) => { const checks = {}; try { // Check database connectivity await db.query('SELECT 1'); checks.database = 'ok'; } catch (err) { checks.database = 'failed'; } try { // Check Redis connectivity await redis.ping(); checks.redis = 'ok'; } catch (err) { checks.redis = 'failed'; } try { // Check that restaurant menu cache is warmed (Swiggy-specific) const cacheReady = await menuCache.isWarm(); checks.menuCache = cacheReady ? 'ok' : 'warming'; } catch (err) { checks.menuCache = 'failed'; } const allReady = Object.values(checks).every(v => v === 'ok'); res.status(allReady ? 200 : 503).json({ checks });}); module.exports = router;SecurityNever expose sensitive information in health endpoints — database connection strings, internal IPs, or stack traces. Health endpoints are often public-facing. Return only status strings and boolean indicators, never raw error objects.
Step 4 — Configure Probes for a Spring Boot Service
Spring Boot has built-in Actuator health endpoints that map perfectly to Kubernetes probes:
# application.yaml — Spring Boot Actuator probe configurationmanagement: endpoint: health: probes: enabled: true # Enable /actuator/health/liveness and /readiness show-details: never # Never expose internals in health response endpoints: web: exposure: include: health, info, prometheus health: livenessstate: enabled: true readinessstate: enabled: true db: enabled: true # Check database connectivity for readiness redis: enabled: true # Check Redis for readiness# deployment-springboot.yaml — probe config for Razorpay payments service (Spring Boot)startupProbe: httpGet: path: /actuator/health/liveness port: 8080 failureThreshold: 60 # Spring Boot with heavy migrations needs up to 10 minutes periodSeconds: 10 livenessProbe: httpGet: path: /actuator/health/liveness port: 8080 periodSeconds: 15 timeoutSeconds: 5 failureThreshold: 3 readinessProbe: httpGet: path: /actuator/health/readiness port: 8080 periodSeconds: 5 timeoutSeconds: 3 failureThreshold: 3 successThreshold: 1TipSpring Boot's
/actuator/health/livenessonly checks the application state (not external dependencies), while/actuator/health/readinesschecks all configured health indicators including database and Redis. This separation maps perfectly to the Kubernetes liveness/readiness model.
Step 5 — Configure Probes for TCP Services (Databases, Kafka)
# StatefulSet probes for PostgreSQL using exec mechanismlivenessProbe: exec: command: - pg_isready - -U - postgres - -h - localhost initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 5 failureThreshold: 6 # Allow 6 failures (60s) before restart — DB restarts are slow readinessProbe: exec: command: - pg_isready - -U - postgres - -h - localhost initialDelaySeconds: 5 periodSeconds: 5 timeoutSeconds: 3 failureThreshold: 3
