CNI

Kubernetes hub

Container Network Interface - a specification and set of plugins that configure network interfaces inside Linux containers, enabling pod-to-pod communication across all nodes in a Kubernetes cluster. Popular implementations include Calico, Cilium, and Flannel. Without a CNI plugin installed, no pods can communicate and all will remain stuck in Pending.

CNI — The Networking Foundation of Every Kubernetes Cluster

What is CNI in Simple Terms?

When Kubernetes creates a pod, it needs to give that pod a unique IP address and connect it to the cluster network so it can talk to every other pod — even on different nodes. CNI is the standard that defines how that networking is set up. Kubernetes itself has no built-in networking — it delegates 100% of pod networking to whichever CNI plugin you install.

Popular CNI plugins used in Indian production clusters:

Plugin	Used By	Key Strength
Calico	Zerodha, banking platforms	Network policy enforcement, BGP routing
Cilium	High-traffic APIs, Hotstar-scale	eBPF-based, no iptables, best performance
Flannel	Dev/test clusters	Simple overlay, easy to set up
Weave	Legacy clusters	Simple mesh networking

How CNI Works — Pod Creation Flow

◈ DIAGRAM

+--------------------------------------------------+
| Pod scheduled on mumbai-prod-node-1              | <- Step 1: Scheduler assigns node
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
| Kubelet creates a network namespace for the pod  | <- Step 2: Isolated network env
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
| Kubelet calls CNI plugin binary (/opt/cni/bin/)  | <- Step 3: CNI takes over
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
| CNI assigns pod IP 10.244.1.15 via IPAM          | <- Step 4: Unique IP allocated
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
| CNI creates veth pair:                           | <- Step 5: Virtual cable created
|   eth0 (inside pod ns) <-> veth0abc (cni0)       |
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
| Routing rules added for 10.244.1.15              | <- Step 6: All nodes can reach pod
+--------------------------------------------------+
                        |
                        v
+--------------------------------------------------+
| Pod can now communicate with every pod           | <- Step 7: Networking ready
+--------------------------------------------------+

What is a veth Pair?

A veth (virtual ethernet) pair is like a virtual network cable with two ends. CNI uses it to connect the isolated pod network namespace to the node's main network:

◈ DIAGRAM

+------------------------------+          +------------------------------+
|            Pod               |          |            Node              |
|                              |          |                              |
|  eth0 (10.244.1.15)          | <------> |  veth0abc  ->  cni0 bridge   |
|  (inside pod namespace)      |          |  (node network namespace)    |
+------------------------------+          +------------------------------+

Traffic leaving the pod goes through eth0 -> veth pair -> node bridge -> routed to destination.

CNI Plugin Comparison — Choosing for Production

Feature	Calico	Cilium	Flannel
Network Policies	Full support	Full support	Not supported
Performance	Good	Excellent (eBPF)	Moderate
Observability	Basic	Deep (Hubble UI)	None
Encryption	WireGuard	WireGuard/IPSec	None
Complexity	Medium	High	Low
Best for	Policy-heavy prod	High-throughput prod	Dev/test

How to Check Which CNI is Running

Bash

# List CNI config files on any worker node
ls /etc/cni/net.d/
 
# Common output examples:
# 10-calico.conflist       (Calico CNI)
# 10-flannel.conflist      (Flannel CNI)
# 05-cilium.conf           (Cilium CNI)
 
# View the full CNI config
cat /etc/cni/net.d/10-calico.conflist
 
# Check CNI plugin binaries installed
ls /opt/cni/bin/

Installing Cilium CNI (Recommended for Production)

Bash

# Install Cilium CLI
curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz
tar xzvf cilium-linux-amd64.tar.gz
sudo mv cilium /usr/local/bin
 
# Pin to a tested version in production — never use @latest on live clusters
# 1.17.0 is an example: check the Cilium releases page for the current stable version instead of hard-coding this one
cilium install --version 1.17.0
 
# Verify all Cilium pods are running
cilium status --wait
 
# Run connectivity test
cilium connectivity test

Troubleshooting CNI Issues

Symptom	Likely Cause	Fix
Pod stuck in `ContainerCreating`	CNI config missing or broken	Check `/etc/cni/net.d/` on the node
All pods `Pending` after fresh cluster	CNI plugin never installed	Install Calico, Cilium, or Flannel
`networkPlugin cni failed to set up pod`	CNI binary missing	Check `/opt/cni/bin/`
Pods can't reach each other cross-node	Pod CIDR routing broken	Verify `kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}'`

Bash

# Describe pod to read CNI error detail
kubectl describe pod <pod-name> -n production
 
# Check CNI plugin logs (Calico example)
kubectl logs -n kube-system -l k8s-app=calico-node --tail=50
 
# Confirm pod CIDRs are assigned per node
kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}'
 
# SSH into node and confirm CNI config exists
ssh rahul@10.0.1.50
ls /etc/cni/net.d/

Tip
For production clusters running high-traffic workloads like Zerodha's trading engine or Hotstar's streaming nodes, use Cilium — it replaces iptables with eBPF for significantly better throughput and built-in observability via the Hubble UI.

Remember
Kubernetes does NOT include any CNI plugin by default. On a bare kubeadm cluster, every pod will stay in Pending indefinitely until you install one. This is the single most common reason a fresh cluster appears broken.

Security
CNI plugins like Calico and Cilium support Kubernetes NetworkPolicy objects that restrict pod-to-pod traffic. Without a NetworkPolicy-capable CNI, any pod in your cluster can freely reach any other pod — a serious risk in multi-tenant environments at Razorpay or PhonePe.

Common Mistake
Switching CNI plugins on a live cluster without draining all nodes first. Migrating from Flannel to Cilium mid-flight will break all existing pod networking and cause a full cluster outage. Always plan CNI migrations during a maintenance window.

Frequently Asked Questions

Why doesn't Kubernetes ship its own built-in networking implementation?

Kubernetes deliberately defines networking as a pluggable interface (CNI) rather than a fixed implementation, because network requirements vary wildly — some clusters need simple flat networking (Flannel), others need network policy enforcement and eBPF-based performance (Cilium), others need integration with existing datacenter routing (Calico with BGP). This plugin model lets cloud providers, on-prem teams, and specialized use cases each pick the CNI that fits their infrastructure instead of Kubernetes forcing one networking model on everyone.

What's the practical symptom when a cluster has no CNI plugin installed, and how do you diagnose it?

Every pod that isn't using host networking gets stuck in `ContainerCreating` or `Pending` indefinitely, and `kubectl describe pod` shows an error like 'network plugin is not ready' or a failure to set up the pod sandbox. This is a common gotcha right after standing up a cluster with `kubeadm` — the control plane comes up fine and nodes show `Ready`, but no application pods will ever start until a CNI plugin (Calico, Cilium, Flannel, etc.) is applied, since kubeadm doesn't install one by default.