Configuring Ingress Controllers with NGINX for Production Traffic
Overview and What You Will Learn
Without an Ingress Controller, exposing services in Kubernetes means creating a separate cloud load balancer for every single service β expensive, unmanageable, and impossible to maintain at scale. This lab walks you through deploying and configuring the NGINX Ingress Controller on a production Kubernetes cluster, setting up path-based and host-based routing, enabling SSL termination, and applying rate limiting to protect your services.
By the end of this guide you will be able to:
- Deploy the NGINX Ingress Controller using Helm on a production cluster
- Configure host-based and path-based routing rules for multiple services
- Terminate SSL using Kubernetes TLS secrets and cert-manager
- Apply rate limiting and connection throttling annotations to protect APIs
- Troubleshoot common Ingress misconfiguration issues with real kubectl commands
Why This Matters in Production
Consider Razorpay running dozens of internal microservices β payments, dashboard, settlements, webhooks, and reporting. Without Ingress, each service needs its own cloud load balancer IP, multiplying infrastructure costs and SSL certificate management overhead. A single NGINX Ingress Controller handles all external traffic through one IP, routes it to the correct service based on hostname or path, terminates SSL centrally, and applies security policies uniformly.
At the scale of platforms like PhonePe or CRED, a misconfigured Ingress can mean routing payment traffic to the wrong backend, or leaving an internal admin service accidentally exposed to the public internet. Getting Ingress right is a fundamental production skill.
Core Principles
How traffic flows through an NGINX Ingress Controller:
External User Request (https://api.razorpay.in/v1/payments) β βΌCloud Load Balancer (single external IP β provisioned by cloud provider) β βΌNGINX Ingress Controller Pod (reads Ingress rules from API server) β βββΊ Host: api.razorpay.in + Path: /v1/payments β βββΊ payments-service:4000 β βββΊ Host: api.razorpay.in + Path: /v1/dashboard β βββΊ dashboard-service:8080 β βββΊ Host: admin.razorpay.in βββΊ admin-service:9000 (internal only)Key components every engineer must understand:
- Ingress Controller β the actual NGINX pod that processes routing rules. Must be deployed separately β Kubernetes does not ship with one by default.
- Ingress Resource β the YAML object that defines the routing rules. Useless without a running controller to read and apply them.
- IngressClass β tells Kubernetes which controller should process a given Ingress resource. Critical when running multiple controllers in one cluster.
- Annotations β NGINX-specific configuration applied per Ingress resource to enable SSL redirect, rate limiting, custom timeouts, and more.
Detailed Step-by-Step Practical Lab
Step 1 β Deploy NGINX Ingress Controller Using Helm
1# Add the official ingress-nginx Helm repository2helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx3helm repo update4 5# Deploy NGINX Ingress Controller into its own namespace6helm install ingress-nginx ingress-nginx/ingress-nginx \7 --namespace ingress-nginx \8 --create-namespace \9 --set controller.replicaCount=2 \10 --set controller.nodeSelector."kubernetes\.io/os"=linux \11 --set controller.service.externalTrafficPolicy=Local12 13# Verify the controller pods are running14kubectl get pods -n ingress-nginx15 16# Get the external IP assigned by the cloud load balancer17kubectl get service ingress-nginx-controller -n ingress-nginx18# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)19# ingress-nginx-controller LoadBalancer 10.96.12.100 34.93.45.201 80:31080/TCP,443:31443/TCPπ Remember: The EXTERNAL-IP shown above is what your DNS records must point to. All your domain names (api.razorpay.in, dashboard.razorpay.in) should have A records pointing to this single IP.Step 2 β Create a Basic Ingress Resource with Path-Based Routing
1# ingress-basic.yaml β route traffic to two backend services2apiVersion: networking.k8s.io/v13kind: Ingress4metadata:5 name: razorpay-api-ingress6 namespace: production7 annotations:8 nginx.ingress.kubernetes.io/ssl-redirect: "true" # Force HTTPS9 nginx.ingress.kubernetes.io/use-regex: "true" # Enable regex paths10 nginx.ingress.kubernetes.io/proxy-body-size: "10m" # Max request body size11 nginx.ingress.kubernetes.io/proxy-connect-timeout: "30" # Connection timeout seconds12 nginx.ingress.kubernetes.io/proxy-read-timeout: "60" # Read timeout seconds13spec:14 ingressClassName: nginx # Must match your installed IngressClass15 rules:16 - host: api.razorpay.in17 http:18 paths:19 - path: /v1/payments20 pathType: Prefix21 backend:22 service:23 name: payments-service24 port:25 number: 400026 - path: /v1/dashboard27 pathType: Prefix28 backend:29 service:30 name: dashboard-service31 port:32 number: 808033 - path: /v1/webhooks34 pathType: Prefix35 backend:36 service:37 name: webhook-service38 port:39 number: 50001kubectl apply -f ingress-basic.yaml2 3# Verify Ingress was created and has an address4kubectl get ingress -n production5# NAME CLASS HOSTS ADDRESS PORTS AGE6# razorpay-api-ingress nginx api.razorpay.in 34.93.45.201 80, 443 2mStep 3 β Enable SSL with cert-manager and Let's Encrypt
1# Install cert-manager for automatic SSL certificate provisioning2helm repo add jetstack https://charts.jetstack.io3helm repo update4 5helm install cert-manager jetstack/cert-manager \6 --namespace cert-manager \7 --create-namespace \8 --set installCRDs=true9 10# Verify cert-manager pods are running11kubectl get pods -n cert-manager1# cluster-issuer.yaml β configure Let's Encrypt production certificate issuer2apiVersion: cert-manager.io/v13kind: ClusterIssuer4metadata:5 name: letsencrypt-production6spec:7 acme:8 server: https://acme-v02.api.letsencrypt.org/directory9 email: rahul@devopsnetwork.in # Must be a real monitored email10 privateKeySecretRef:11 name: letsencrypt-production-key12 solvers:13 - http01:14 ingress:15 class: nginx1kubectl apply -f cluster-issuer.yaml1# ingress-ssl.yaml β Ingress with automatic SSL certificate2apiVersion: networking.k8s.io/v13kind: Ingress4metadata:5 name: razorpay-api-ingress6 namespace: production7 annotations:8 nginx.ingress.kubernetes.io/ssl-redirect: "true"9 cert-manager.io/cluster-issuer: "letsencrypt-production" # Auto-provision SSL10spec:11 ingressClassName: nginx12 tls:13 - hosts:14 - api.razorpay.in15 - dashboard.razorpay.in16 secretName: razorpay-tls-secret # cert-manager stores the cert here17 rules:18 - host: api.razorpay.in19 http:20 paths:21 - path: /22 pathType: Prefix23 backend:24 service:25 name: payments-service26 port:27 number: 400028 - host: dashboard.razorpay.in29 http:30 paths:31 - path: /32 pathType: Prefix33 backend:34 service:35 name: dashboard-service36 port:37 number: 80801kubectl apply -f ingress-ssl.yaml2 3# Watch the SSL certificate being issued β takes 60-120 seconds4kubectl get certificate -n production -w5# NAME READY SECRET AGE6# razorpay-tls-secret True razorpay-tls-secret 90sπ‘ Tip: If the certificate stays inFalsestate for more than 5 minutes, check the CertificateRequest and Order objects:kubectl describe certificaterequest -n productionβ it will show exactly which ACME challenge step failed.
Step 4 β Apply Rate Limiting to Protect APIs
1# ingress-ratelimit.yaml β protect the payments API from abuse2apiVersion: networking.k8s.io/v13kind: Ingress4metadata:5 name: payments-ingress-protected6 namespace: production7 annotations:8 nginx.ingress.kubernetes.io/ssl-redirect: "true"9 cert-manager.io/cluster-issuer: "letsencrypt-production"10 11 # Rate limiting β critical for payment APIs12 nginx.ingress.kubernetes.io/limit-rps: "20" # Max 20 requests/second per IP13 nginx.ingress.kubernetes.io/limit-connections: "10" # Max 10 concurrent connections per IP14 nginx.ingress.kubernetes.io/limit-burst-multiplier: "5" # Allow bursts up to 100 rps briefly15 16 # Security headers17 nginx.ingress.kubernetes.io/configuration-snippet: |18 more_set_headers "X-Frame-Options: DENY";19 more_set_headers "X-Content-Type-Options: nosniff";20 more_set_headers "Referrer-Policy: strict-origin-when-cross-origin";21spec:22 ingressClassName: nginx23 tls:24 - hosts:25 - api.razorpay.in26 secretName: razorpay-tls-secret27 rules:28 - host: api.razorpay.in29 http:30 paths:31 - path: /v1/payments32 pathType: Prefix33 backend:34 service:35 name: payments-service36 port:37 number: 4000Step 5 β Configure Host-Based Routing for Multiple Domains
1# ingress-multidomain.yaml β separate domains to separate services2apiVersion: networking.k8s.io/v13kind: Ingress4metadata:5 name: razorpay-multidomain-ingress6 namespace: production7 annotations:8 nginx.ingress.kubernetes.io/ssl-redirect: "true"9 cert-manager.io/cluster-issuer: "letsencrypt-production"10spec:11 ingressClassName: nginx12 tls:13 - hosts:14 - api.razorpay.in15 - admin.razorpay.in16 - webhooks.razorpay.in17 secretName: razorpay-multidomain-tls18 rules:19 - host: api.razorpay.in # Public API β internet facing20 http:21 paths:22 - path: /23 pathType: Prefix24 backend:25 service:26 name: public-api-service27 port:28 number: 400029 30 - host: admin.razorpay.in # Admin panel β restrict via firewall rules31 http:32 paths:33 - path: /34 pathType: Prefix35 backend:36 service:37 name: admin-service38 port:39 number: 900040 41 - host: webhooks.razorpay.in # Webhook receiver β high timeout needed42 http:43 paths:44 - path: /45 pathType: Prefix46 backend:47 service:48 name: webhook-service49 port:50 number: 5000β οΈ Security: Never expose admin dashboards through a public Ingress without IP whitelisting. Add nginx.ingress.kubernetes.io/whitelist-source-range: "10.0.0.0/8,YOUR_OFFICE_IP/32" to the admin Ingress annotation block to restrict access to known IPs only.Step 6 β Verify and Troubleshoot the Ingress
1# Check Ingress resource status and backend addresses2kubectl describe ingress razorpay-api-ingress -n production3 4# Check NGINX Ingress Controller logs for routing errors5kubectl logs -n ingress-nginx \6 -l app.kubernetes.io/name=ingress-nginx \7 --tail=100 -f8 9# Test routing from inside the cluster using a debug pod10kubectl run curl-test --image=curlimages/curl -it --rm -n production -- sh11 12# Inside the debug pod β test internal routing13curl -H "Host: api.razorpay.in" http://10.96.12.100/v1/paymentsProduction Best Practices & Common Pitfalls
- Always run at least 2 replicas of the Ingress Controller with a PodDisruptionBudget. A single controller pod is a single point of failure for all cluster traffic.
- Use
externalTrafficPolicy: Localon the Ingress Controller service to preserve the real client IP in access logs β essential for rate limiting and security auditing. - Set resource requests and limits on the Ingress Controller pods. An unthrottled controller under traffic spike can OOMKill and drop all traffic simultaneously.
- Always use
IngressClassexplicitly. Relying on the legacykubernetes.io/ingress.classannotation causes unpredictable behaviour in newer Kubernetes versions. - Separate Ingress resources per service rather than one monolithic Ingress. Smaller resources are easier to audit, roll back, and debug independently.
π΄ Common Mistake: Creating an Ingress resource without installing an Ingress Controller first. The Ingress object will be accepted by the API server with no errors but traffic will never be routed. Always confirm the controller is running with kubectl get pods -n ingress-nginx before creating Ingress resources.Quick Reference & Troubleshooting Commands
| Command | Purpose |
|---|---|
kubectl get ingress -n <ns> |
List all Ingress resources and their addresses |
kubectl describe ingress <name> -n <ns> |
Full Ingress config with backend endpoint status |
kubectl get ingressclass |
List available IngressClass controllers in cluster |
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx |
NGINX controller logs for routing errors |
kubectl get certificate -n <ns> |
SSL certificate issuance status |
kubectl describe certificaterequest -n <ns> |
Debug failed SSL certificate provisioning |
kubectl get events -n ingress-nginx --sort-by='.lastTimestamp' |
Ingress controller event timeline |
helm upgrade ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx |
Upgrade NGINX Ingress Controller version |
kubectl exec -it <nginx-pod> -n ingress-nginx -- nginx -t |
Validate generated NGINX config inside controller |