ResourceQuota

Kubernetes hub

A namespace-scoped Kubernetes object that enforces hard upper limits on the total amount of compute resources (CPU, memory) and API objects (pods, secrets, services) that can exist within a namespace. ResourceQuotas protect shared clusters from a single team or runaway workload consuming all available resources.

ResourceQuota — Capping What Each Team Can Consume

Why This Matters in Shared Clusters

Without quotas on a multi-tenant cluster, one team running a memory leak can OOMKill every other team's pods by exhausting node memory. At Razorpay, where multiple teams share the same production cluster, ResourceQuotas are the enforcement boundary between teams.

◈ DIAGRAM

+----------------------------------------------------+
| mumbai-prod-cluster  (total: 128 CPU, 512Gi RAM)   |
+----------------------------------------------------+
        |               |               |
        v               v               v
+---------------+ +---------------+ +---------------+
| payments-team | | risk-team     | | data-team     |
| namespace     | | namespace     | | namespace     |
|               | |               | |               |
| Quota:        | | Quota:        | | Quota:        |
| 32 CPU        | | 16 CPU        | | 64 CPU        |
| 128Gi RAM     | | 64Gi RAM      | | 256Gi RAM     |
| 100 pods      | | 50 pods       | | 200 pods      |
+---------------+ +---------------+ +---------------+

A Complete ResourceQuota Manifest

YAML

apiVersion: v1
kind: ResourceQuota
metadata:
  name: payments-team-quota
  namespace: payments-prod
spec:
  hard:
    # ── Compute Resources ──
    requests.cpu: "32"               # Total CPU requests across ALL pods in namespace
    requests.memory: 128Gi           # Total memory requests across ALL pods
    limits.cpu: "64"                 # Total CPU limits across ALL pods
    limits.memory: 256Gi             # Total memory limits across ALL pods
    # ── Object Count Limits ──
    pods: "100"                      # Max number of pods
    services: "20"                   # Max number of Services
    secrets: "50"                    # Max number of Secrets
    configmaps: "30"                 # Max number of ConfigMaps
    persistentvolumeclaims: "20"     # Max number of PVCs
    # ── Service Type Limits ──
    services.loadbalancers: "3"      # Limit expensive cloud load balancers
    services.nodeports: "0"          # Block NodePort services entirely

What Happens When a Quota Is Hit

Bash

+------------------------------------------+
| kubectl apply -f new-deployment.yaml     | <- Team tries to deploy
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| Admission Controller checks quota        | <- Checks: used + requested <= hard
+------------------------------------------+
            |               |
            v               v
+------------------+    +------------------------------------------+
| UNDER QUOTA      |    | QUOTA EXCEEDED                           |
| Pod created OK   |    | Error: exceeded quota: payments-team-    |
|                  |    | quota, requested: requests.memory=4Gi,   |
|                  |    | used: requests.memory=126Gi,             |
|                  |    | limited: requests.memory=128Gi           |
+------------------+    +------------------------------------------+

Bash

# What the error looks like in practice
kubectl apply -f new-deployment.yaml
# Error from server (Forbidden): pods "api-server-xyz" is forbidden:
# exceeded quota: payments-team-quota, requested: requests.memory=4Gi,
# used: requests.memory=126Gi, limited: requests.memory=128Gi

The pod goes into Pending and won't schedule until other pods are removed or the quota is raised.

ResourceQuota Requires Resource Requests on Every Pod

Remember
If a namespace has a ResourceQuota for CPU or memory, every pod in that namespace MUST specify resource requests and limits. Pods without requests will be rejected outright — even if the namespace has plenty of quota headroom remaining.

YAML

# This pod will be REJECTED if the namespace has a CPU/memory quota
containers:
  - name: api
    image: api:latest
    # Missing resources block -> rejected with "must specify requests" error
 
# This pod will be ACCEPTED
containers:
  - name: api
    image: api:latest
    resources:
      requests:
        cpu: "500m"
        memory: "512Mi"
      limits:
        cpu: "1"
        memory: "1Gi"

Viewing Quota Usage

Bash

# Summary view — all quotas in a namespace
kubectl get quota -n payments-prod
 
# Detailed usage breakdown — see used vs hard for every resource
kubectl describe quota payments-team-quota -n payments-prod
# Resource                Used   Hard
# --------                ----   ----
# limits.cpu              28     64
# limits.memory           96Gi   256Gi
# persistentvolumeclaims  14     20
# pods                    67     100
# requests.cpu            14     32
# requests.memory         48Gi   128Gi
# secrets                 31     50
# services                12     20
# services.loadbalancers  2      3
# services.nodeports      0      0
 
# Check quota across all namespaces at once
kubectl get quota -A

ResourceQuota vs LimitRange — How They Work Together

◈ DIAGRAM

+------------------------------------------+
| ResourceQuota                            | <- Namespace ceiling:
|                                          |    "This namespace gets 32 CPU total"
| Enforced at: admission time              |
| Scope: entire namespace                  |
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| LimitRange                               | <- Per-container guardrails:
|                                          |    "Each container: 100m-4 CPU"
| Enforced at: admission time              |
| Scope: individual container              |
+------------------------------------------+

Use both together. ResourceQuota without LimitRange means a single container can claim all 32 CPU in one pod. LimitRange without ResourceQuota means per-container limits are set but the namespace has no ceiling — 1000 small pods could still exhaust the cluster.

Troubleshooting Common ResourceQuota Problems

Problem	Symptom	Fix
New pods rejected with Forbidden	`exceeded quota` error on `kubectl apply`	Check `kubectl describe quota` to see which resource is exhausted — scale down unused deployments or request a quota increase
Pod rejected with "must specify requests"	`failed quota` even when under limits	Namespace has ResourceQuota but pod spec has no `resources` block — add explicit CPU and memory requests
Quota shows 0 used but pods exist	Old quota object not matching namespace	Quota `metadata.namespace` doesn't match the namespace you're deploying into — verify with `kubectl get quota -A`
Services.nodeports: 0 but NodePort needed	Service creation blocked	Request a quota change or use an Ingress instead of NodePort — NodePorts are blocked intentionally in multi-tenant clusters
Quota raised but pod still pending	Pod stuck after quota increase	The pod was rejected before the quota increase — delete and reapply the pod after the quota is updated

Tip
Set up a Prometheus alert when any namespace hits 80% of its quota. At 100%, new deployments silently fail — which is very confusing during an incident when you're trying to scale up your pods to handle a traffic spike.

Security
Use services.nodeports: "0" in production quotas to block NodePort services across all team namespaces. NodePorts expose services directly on the node's public IP — on a shared cluster like Razorpay's, this bypasses all ingress-level authentication and rate limiting.

Common Mistake
Setting a quota only on requests.cpu and requests.memory but not on limits.cpu and limits.memory. A developer can set requests: 100m but limits: 64 CPU on a single container — the quota passes (requests are under the cap), but the container can burst and consume the entire node.

Frequently Asked Questions

Does a ResourceQuota work if individual pods don't specify resource requests?

No — if a ResourceQuota sets limits on `requests.cpu` or `requests.memory` for a namespace, Kubernetes requires every pod created in that namespace to explicitly declare those requests, and will reject pod creation otherwise. This is often the real-world trigger for teams to finally add resource requests across their manifests: the quota enforcement forces the discipline that best practice alone didn't.

What's a common surprise when a namespace hits its ResourceQuota limit?

New pod creation starts failing outright with a quota-exceeded admission error — it doesn't queue or throttle gracefully, it just rejects the request. This catches teams off guard during a deploy or a HorizontalPodAutoscaler scale-up event when the namespace is already near its limit. Pair ResourceQuotas with monitoring on quota utilization (not just node utilization) so you get warned before a deploy fails at the worst possible moment.