DaemonSet

DaemonSet — One Agent Per Node, Always

What is a DaemonSet in Simple Terms?

A DaemonSet is a standing order to the scheduler: "This pod must always run on every node — one copy per node, no more, no less." Not one total replica. One per node. When a new node joins the cluster, the DaemonSet pod is automatically placed on it. When a node is removed, the pod is cleaned up automatically.

DaemonSet vs Deployment — The Key Difference

One Pod Per Node — How It Looks

When to Use a DaemonSet

Use DaemonSet for infrastructure agents:

• Log collection — Fluentd, Filebeat, Promtail (must read log files from every node's disk)
• Metrics scraping — Node Exporter (must collect CPU/memory/disk from every node)
• Network plugins — Calico, Cilium CNI agents (must configure networking on every node)
• Security agents — Falco, CrowdStrike (must inspect every node's syscalls and processes)
• Storage drivers — CSI node drivers that attach and mount volumes

Do NOT use DaemonSet for:

• Application workloads (use Deployment)
• Batch processing (use Jobs or CronJobs)
• Anything where you want explicit control over replica count

A Real DaemonSet — Node Exporter for Prometheus

Targeting Specific Nodes — Not Always Every Node

You do not always want a DaemonSet on every node. A GPU monitoring agent should only run on GPU nodes:

For more complex targeting, use nodeAffinity:

Tolerations — Getting onto Tainted Nodes

Control-plane nodes carry a default taint: node-role.kubernetes.io/control-plane:NoSchedule. Without a matching toleration, your DaemonSet will skip them. The operator: Exists toleration bypasses ALL taints — use it for critical agents that must run everywhere like Falco or CNI plugins.

Key DaemonSet Commands

⚠️ Security: Setting hostNetwork: true and hostPID: true gives the container full visibility into the host's network stack and every process running on the node. Only use these flags for trusted infrastructure agents like Node Exporter or Falco — never for application workloads. In Hotstar or PhonePe production clusters, DaemonSet pods with host access should be reviewed as part of every security audit.

📌 Remember: DaemonSet pod count equals node count. If your cluster has 12 nodes and your DaemonSet shows 10 pods, two nodes have issues — either they are tainted without a matching toleration, or the pods are failing on those nodes. Use kubectl get pods -o wide to identify which nodes are missing coverage.

🔴 Common Mistake: Using a Deployment with replicas matching your node count as a substitute for a DaemonSet. If a node is added later, the Deployment will not automatically place a pod on it — you are back to manual scaling. Always use a DaemonSet for anything that must run on every node.

💡 Tip: In Zerodha or Swiggy-scale clusters, DaemonSets for log collection (Fluentd/Promtail) are often the highest-volume pods in the cluster. Set proper resources.requests and resources.limits on them — an unthrottled Fluentd pod can consume enough CPU to starve application pods on the same node during log bursts.