HPA

Kubernetes hub

Horizontal Pod Autoscaler - a Kubernetes controller that automatically scales the number of pod replicas in a Deployment or StatefulSet up or down based on observed CPU utilization, memory usage, or custom application metrics. It eliminates the need to manually adjust replica counts during traffic spikes and quiet periods.

HPA - Automatic Scaling Based on Real Traffic

What is HPA in Simple Terms?

HPA is your automatic traffic manager. During Swiggy's dinner rush (7pm–9pm), order volume spikes 10x. HPA detects that CPU is hitting 80%, automatically adds more pods to handle the load, then scales back down at midnight when traffic drops — saving infrastructure costs without any human intervention.

How HPA Decides to Scale

◈ DIAGRAM

+--------------------------------------------------+
| HPA controller polls Metrics Server every 15s   | <- continuous monitoring
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
| Current avg CPU across pods: 85%                |
| Target avg CPU configured: 70%                  |
| Pods needed = ceil(current / target * replicas) |
+--------------------------------------------------+
                        |
            +-----------+-----------+
            |                       |
            v                       v
+---------------------+   +---------------------+
| CPU > target        |   | CPU < target         |
| -> Scale UP         |   | -> Scale DOWN        |
| Add pods (up to max)|   | Remove pods (to min) |
+---------------------+   +---------------------+

The scale formula Kubernetes uses: desiredReplicas = ceil(currentReplicas * (currentMetric / targetMetric))

Example HPA Manifest

YAML

# hpa.yaml — autoscale the order-service based on CPU and memory
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: order-service-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: order-service
  minReplicas: 3         # Always keep at least 3 pods — never scale to zero
  maxReplicas: 20        # Hard ceiling — prevents runaway scaling
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70    # Scale when avg CPU across all pods hits 70%
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80    # Also scale if memory pressure builds up

Deployment Must Have Resource Requests Defined

HPA cannot calculate utilization percentage without knowing what the pod requested. This is a mandatory prerequisite:

YAML

# deployment.yaml — resources.requests MUST be set for HPA to work
spec:
  containers:
    - name: order-service
      image: registry.swiggy.in/order-service:v3.1.0
      resources:
        requests:
          cpu: "250m"        # HPA uses this as the baseline for % calculation
          memory: "256Mi"
        limits:
          cpu: "1000m"
          memory: "512Mi"

Checking HPA Status

Bash

# See current replica count, targets, and scaling activity
kubectl get hpa -n production
 
# Output:
# NAME                REFERENCE               TARGETS         MINPODS   MAXPODS   REPLICAS
# order-service-hpa   Deployment/order-svc    68%/70%         3         20        8
 
# Detailed status including last scale event
kubectl describe hpa order-service-hpa -n production
 
# Watch HPA react to live traffic in real time
kubectl get hpa order-service-hpa -n production -w

Scaling Behavior — Controlling Scale Up and Scale Down Speed

By default, HPA can be aggressive on scale-up and slow on scale-down. You can tune this:

YAML

# Add behavior block to control scaling velocity
spec:
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0      # Scale up immediately when needed
      policies:
        - type: Pods
          value: 4                        # Add at most 4 pods per scaling event
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300    # Wait 5 minutes before scaling down
      policies:
        - type: Percent
          value: 10                       # Remove at most 10% of pods per minute
          periodSeconds: 60

The stabilizationWindowSeconds on scale-down prevents flapping — HPA won't remove pods immediately after a traffic spike drops, giving headroom for the next wave.

Troubleshooting HPA

Symptom	Likely Cause	Fix
HPA shows `<unknown>/70%`	Metrics Server not installed	Install `metrics-server` in kube-system
HPA not scaling up	`resources.requests` not set on deployment	Add CPU/memory requests to pod spec
HPA stuck at `minReplicas`	Current CPU below threshold	Check actual usage with `kubectl top pods`
HPA scaling too aggressively	No stabilization window	Add `behavior.scaleDown.stabilizationWindowSeconds`
Replicas hit `maxReplicas` and stop	Max ceiling reached	Raise `maxReplicas` or investigate pod performance

Bash

# Verify Metrics Server is working (prerequisite for HPA)
kubectl top pods -n production
 
# Check why HPA is not scaling — events section is key
kubectl describe hpa order-service-hpa -n production | grep -A 20 Events
 
# Manually simulate load to test HPA behavior
kubectl run load-gen --image=busybox -it --rm -- \
  /bin/sh -c "while true; do wget -q -O- http://order-service.production; done"

Remember
HPA requires the Metrics Server to be installed in the cluster. Without it, HPA cannot read CPU or memory metrics and will report <unknown> targets — staying completely inactive. Run kubectl top pods to verify Metrics Server is working before creating any HPA.

Common Mistake
Setting minReplicas: 1 in production. If that single pod is being replaced during a scale-up event, your service has zero availability for the seconds it takes to start a new pod. Always set minReplicas: 3 or higher for any production workload at Razorpay or PhonePe scale.

Tip
Set scaleDown.stabilizationWindowSeconds: 300 in the HPA behavior block. Without this, HPA will remove pods immediately after a traffic spike drops — only to add them back 2 minutes later when the next spike arrives. The stabilization window prevents this flapping and keeps your pod count stable during volatile traffic patterns like Hotstar's live streaming events.

Security
HPA with maxReplicas set too high can become a cost explosion vector. A traffic spike — or a DDoS — can trigger HPA to spin up hundreds of pods, consuming all cluster node capacity and triggering expensive cloud autoscaling. Always set a sensible maxReplicas ceiling and configure cluster-level resource quotas per namespace to cap total pod resource consumption.

Frequently Asked Questions

What metrics can the Horizontal Pod Autoscaler actually scale on beyond CPU?

HPA natively supports CPU and memory via the metrics-server, but production setups often need more: with the custom metrics API (backed by Prometheus Adapter, for example) it can scale on request latency, queue depth, or requests-per-second. HPA polls metrics every 15 seconds by default and computes desired replicas as ceil(currentReplicas * currentMetric / desiredMetric), then applies stabilization windows to avoid flapping.

Why does HPA sometimes fail to scale even though the app is clearly overloaded?

The most common cause is pods without CPU requests set — HPA's percentage-based targets are meaningless without a request baseline to measure against, so it silently does nothing. Another frequent gotcha: scaling on CPU while the real bottleneck is downstream (a database or external API), so new pods spin up but throughput doesn't improve. Pair HPA with sensible requests and, ideally, a metric that reflects actual load, not just CPU.