Learn how to secure Kubernetes clusters end to end - covering RBAC and least privilege, Pod Security Standards, Network Policies, Secrets encryption at rest, service account hardening, Falco runtime security, and etcd protection.
Kubernetes is not a single system — it is an API server, a scheduler, a controller manager, a key-value store (etcd), a container runtime, a network layer, and potentially hundreds of running workloads, all connected. Each of these components has its own attack surface. The most common Kubernetes security incidents follow predictable patterns: **Overprivileged service accounts.** A developer creates a service account for a deployment that needs to read ConfigMaps. They give it `cluster-admin` because it is easier than figuring out the exact permissions. That service account token, if stolen by a compromised pod, gives an attacker full cluster control. **Pods running as root.** A container runs as root because the base image defaults to root. If the container is compromised, the attacker has root inside the container — and depending on the host configuration, may be able to escape to the node. **No network policies.** All pods can talk to all other pods by default. A compromised pod can probe every service in the cluster, exfiltrate data from databases, and move laterally to more sensitive workloads. **Unencrypted secrets in etcd.** Kubernetes Secrets are base64-encoded by default, not encrypted. Anyone with access to the etcd database can read every secret in the cluster. This module covers the controls that address each of these patterns. ---
RBAC is the primary mechanism for controlling who can do what in a Kubernetes cluster. Understanding it properly is essential because getting it wrong creates either over-permissive access (security risk) or under-permissive access (operational problems). ### The RBAC Model ``` Four objects in the RBAC system: Role / ClusterRole What can be done — a set of permissions Role = namespaced (only within one namespace) ClusterRole = cluster-wide (across all namespaces, or for cluster-scoped resources) RoleBinding / ClusterRoleBinding Who can do it — binds a Role to a subject RoleBinding = in a specific namespace ClusterRoleBinding = cluster-wide ``` ### Create a Least-Privilege Role A developer needs to read Pods and their logs in the `production` namespace. Nothing else. ```yaml # read-pods-role.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: pod-reader namespace: production rules: - apiGroups: [""] # "" means the core API group resources: ["pods"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["pods/log"] # Pod logs are a subresource verbs: ["get"] ``` Bind it to a developer: ```yaml # developer-binding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: developer-pod-reader namespace: production subjects: - kind: User name: jane@company.com # Authenticated user identity apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io ``` Apply and verify: ```bash kubectl apply -f read-pods-role.yaml kubectl apply -f developer-binding.yaml # Test what jane can do kubectl auth can-i get pods --namespace=production --as=jane@company.com # Output: yes kubectl auth can-i delete pods --namespace=production --as=jane@company.com # Output: no kubectl auth can-i get secrets --namespace=production --as=jane@company.com # Output: no ``` ### Service Account RBAC — The Most Important Use Case Every pod runs as a service account. That service account is given a token mounted at `/var/run/secrets/kubernetes.io/serviceaccount/token`. If a pod is compromised, the attacker has that token and can use it to call the Kubernetes API. **The critical mistake**: using the default service account or creating service accounts with ClusterAdmin. ```yaml # Correct approach: create a dedicated service account with minimal permissions # 1. Create a dedicated service account apiVersion: v1 kind: ServiceAccount metadata: name: payment-service-account namespace: payments automountServiceAccountToken: false # Do not auto-mount — only mount if needed --- # 2. Create a role with only what this service needs apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: payment-service-role namespace: payments rules: - apiGroups: [""] resources: ["configmaps"] resourceNames: ["payment-config"] # Only this specific ConfigMap verbs: ["get"] - apiGroups: [""] resources: ["secrets"] resourceNames: ["payment-tls-cert"] # Only this specific Secret verbs: ["get"] --- # 3. Bind the role to the service account apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: payment-service-binding namespace: payments subjects: - kind: ServiceAccount name: payment-service-account namespace: payments roleRef: kind: Role name: payment-service-role apiGroup: rbac.authorization.k8s.io --- # 4. Use the service account in the deployment apiVersion: apps/v1 kind: Deployment metadata: name: payment-service namespace: payments spec: template: spec: serviceAccountName: payment-service-account automountServiceAccountToken: true # Only because this pod needs it ``` ### RBAC Hardening Rules ``` 1. Never use cluster-admin for workloads The cluster-admin ClusterRole has permission to do ANYTHING Only humans doing cluster administration should have this 2. Avoid wildcards in rules BAD: verbs: ["*"] resources: ["*"] GOOD: verbs: ["get","list"] resources: ["configmaps"] 3. Prefer namespace-scoped roles over cluster roles RoleBinding limits blast radius to one namespace ClusterRoleBinding affects the entire cluster 4. Disable service account token auto-mounting by default Set automountServiceAccountToken: false at the service account level Enable explicitly only for pods that need Kubernetes API access 5. Regularly audit RBAC permissions kubectl get clusterrolebindings -o wide | grep -v system: kubectl get rolebindings --all-namespaces | grep -v system: ``` ---
Pod Security Standards replace the deprecated PodSecurityPolicy. They define three levels of security that can be enforced at the namespace level. This is the built-in mechanism for ensuring pods cannot run as root, cannot use privileged mode, and must respect security context constraints. ### The Three Levels ``` Privileged — No restrictions Use for: system components, trusted infrastructure tools (Falco, CNI plugins) Allows: privileged containers, host network, host PID, any user Baseline — Basic protections Use for: most workloads that need minor permissions Blocks: privileged containers, host namespaces, dangerous capabilities Allows: root user, basic capabilities, host ports Restricted — Hardened Use for: production application workloads Requires: non-root user, no privilege escalation, seccomp profile, dropped capabilities Most secure, some apps may need adjustment to comply ``` ### Apply Pod Security Admission to Namespaces ```bash # Label namespaces to enforce security levels # The label format is: pod-security.kubernetes.io/<MODE>: <LEVEL> # Production namespace — enforce restricted (block non-compliant pods) kubectl label namespace production \ pod-security.kubernetes.io/enforce=restricted \ pod-security.kubernetes.io/enforce-version=latest \ pod-security.kubernetes.io/warn=restricted \ pod-security.kubernetes.io/audit=restricted # Staging namespace — warn but don't block kubectl label namespace staging \ pod-security.kubernetes.io/warn=restricted \ pod-security.kubernetes.io/audit=restricted # System namespace — privileged (system components need it) kubectl label namespace kube-system \ pod-security.kubernetes.io/enforce=privileged ``` ### Write Pods That Pass the Restricted Profile A pod spec that passes the `restricted` Pod Security Standard: ```yaml apiVersion: v1 kind: Pod metadata: name: secure-app namespace: production spec: securityContext: runAsNonRoot: true # Must not run as root runAsUser: 1000 # Explicit non-root UID runAsGroup: 1000 fsGroup: 1000 # Volume files owned by this GID seccompProfile: type: RuntimeDefault # Apply default seccomp profile containers: - name: app image: myapp:latest securityContext: allowPrivilegeEscalation: false # Cannot gain new privileges readOnlyRootFilesystem: true # Read-only root filesystem capabilities: drop: ["ALL"] # Drop all Linux capabilities # Only add back specifically what is needed: # add: ["NET_BIND_SERVICE"] # Only if binding to port < 1024 resources: requests: cpu: "100m" memory: "128Mi" limits: cpu: "500m" memory: "512Mi" volumeMounts: - name: tmp mountPath: /tmp # App needs a writable /tmp volumes: - name: tmp emptyDir: {} # Temporary writable volume ``` ### Testing PSS Compliance ```bash # Dry-run to check if a pod would be blocked kubectl apply --dry-run=server -f my-pod.yaml --namespace=production # If the pod violates the restricted profile, you will see: # Warning: would violate PodSecurity "restricted:latest" # Error: pods "my-pod" is forbidden: violates PodSecurity "restricted:latest" # Check what level a namespace enforces kubectl get namespace production -o jsonpath='{.metadata.labels}' | python3 -m json.tool ``` ---
By default, Kubernetes allows all pods to communicate with all other pods in the cluster. Network Policies allow you to define exactly which pods can communicate with which. Without them, a compromised pod has free access to your entire internal network. ### The Default Deny Baseline Start by denying all traffic, then explicitly allow what is needed: ```yaml # default-deny.yaml — apply to every production namespace apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: production spec: podSelector: {} # Empty selector matches ALL pods in namespace policyTypes: - Ingress - Egress ``` After applying this, no pod in the `production` namespace can receive or send any network traffic. Now add back what is needed. ### Allow Specific Traffic Flows ```yaml # Allow the frontend to talk to the backend apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-frontend-to-backend namespace: production spec: podSelector: matchLabels: app: backend # This policy applies to backend pods policyTypes: - Ingress ingress: - from: - podSelector: matchLabels: app: frontend # Only allow traffic from frontend pods ports: - protocol: TCP port: 8080 ``` ```yaml # Allow the backend to connect to the database apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-backend-to-db namespace: production spec: podSelector: matchLabels: app: database policyTypes: - Ingress ingress: - from: - podSelector: matchLabels: app: backend ports: - protocol: TCP port: 5432 ``` ```yaml # Allow DNS resolution (egress to kube-dns on port 53) # Without this, pods cannot resolve any hostnames apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-dns-egress namespace: production spec: podSelector: {} policyTypes: - Egress egress: - to: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: kube-system ports: - protocol: UDP port: 53 - protocol: TCP port: 53 ``` ### Cross-Namespace Network Policies ```yaml # Allow monitoring namespace to scrape metrics from production apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-monitoring-scrape namespace: production spec: podSelector: matchLabels: metrics: "true" # Pods that expose metrics policyTypes: - Ingress ingress: - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: monitoring podSelector: matchLabels: app: prometheus # Only from Prometheus specifically ports: - protocol: TCP port: 9090 ``` ### Verifying Network Policies ```bash # List all network policies in a namespace kubectl get networkpolicies --namespace=production # Describe a specific policy kubectl describe networkpolicy default-deny-all --namespace=production # Test connectivity (from inside a pod) kubectl exec -it frontend-pod -- curl http://backend-service:8080/health # Should work if policy allows it kubectl exec -it frontend-pod -- curl http://database-service:5432 # Should fail if no policy allows frontend → database ``` ---
Kubernetes Secrets are base64-encoded by default — not encrypted. Base64 is encoding, not encryption. Anyone with read access to etcd can decode every secret in the cluster with a single command. ### How Unencrypted Secrets Are Stored ```bash # Check if encryption is configured kubectl get configmap -n kube-system kube-apiserver -o yaml | grep encryption # If empty — secrets are stored in plain text in etcd # What an unprotected secret looks like in etcd ETCDCTL_API=3 etcdctl get /registry/secrets/default/my-secret # Output shows base64-encoded data — trivially decodable ``` ### Enable Encryption at Rest Create an encryption configuration file on the control plane node: ```yaml # /etc/kubernetes/enc/enc.yaml apiVersion: apiserver.config.k8s.io/v1 kind: EncryptionConfiguration resources: - resources: - secrets - configmaps # Optional: encrypt ConfigMaps too providers: - aescbc: # AES-CBC encryption keys: - name: key1 secret: <BASE64-ENCODED-32-BYTE-KEY> # Generate: head -c 32 /dev/urandom | base64 - identity: {} # Fallback for reading unencrypted secrets during migration ``` Generate the encryption key: ```bash # Generate a strong 32-byte key head -c 32 /dev/urandom | base64 # Save this somewhere secure — if lost, all encrypted data is unrecoverable ``` Configure the API server to use it by editing `/etc/kubernetes/manifests/kube-apiserver.yaml`: ```yaml spec: containers: - command: - kube-apiserver - --encryption-provider-config=/etc/kubernetes/enc/enc.yaml # Add this volumeMounts: - name: enc mountPath: /etc/kubernetes/enc readOnly: true volumes: - name: enc hostPath: path: /etc/kubernetes/enc type: DirectoryOrCreate ``` After restarting the API server, encrypt existing secrets: ```bash # Force re-encrypt all existing secrets kubectl get secrets --all-namespaces -o json | kubectl replace -f - # Verify encryption — output should start with "k8s:enc:aescbc:v1:" ETCDCTL_API=3 etcdctl get /registry/secrets/default/my-secret | hexdump -C | head -5 ``` ### Using External KMS for Better Key Protection For production, encryption keys should not be stored on the control plane node. Use a KMS provider: ```yaml apiVersion: apiserver.config.k8s.io/v1 kind: EncryptionConfiguration resources: - resources: - secrets providers: - kms: apiVersion: v2 name: aws-kms-provider # Uses AWS KMS, Azure Key Vault, or GCP KMS endpoint: unix:///tmp/kms.socket cachesize: 1000 - identity: {} ``` With KMS, the encryption keys live in a hardware-backed key management service outside your cluster. Compromising etcd alone is not enough — the attacker also needs access to the KMS. ---
Service accounts are the identities that pods use to authenticate to the Kubernetes API. They are one of the most common privilege escalation vectors. ### Disable Auto-Mounting by Default Every pod automatically gets a service account token mounted unless you disable it. This token can be used to call the Kubernetes API: ```yaml # Disable auto-mounting at the service account level apiVersion: v1 kind: ServiceAccount metadata: name: my-service namespace: production automountServiceAccountToken: false # No token unless explicitly requested ``` ```yaml # Pods that genuinely need API access must explicitly request the token apiVersion: v1 kind: Pod metadata: name: api-client spec: serviceAccountName: my-service automountServiceAccountToken: true # Explicit opt-in ``` ### Use Projected Tokens with Short Expiry Instead of long-lived service account tokens (which never expire), use projected volumes with time-bound tokens: ```yaml apiVersion: v1 kind: Pod metadata: name: api-client spec: serviceAccountName: my-service containers: - name: app image: myapp:latest volumeMounts: - name: kube-api-access mountPath: /var/run/secrets/kubernetes.io/serviceaccount readOnly: true volumes: - name: kube-api-access projected: sources: - serviceAccountToken: path: token expirationSeconds: 3600 # Token expires in 1 hour audience: "https://kubernetes.default.svc" - configMap: name: kube-root-ca.crt items: - key: ca.crt path: ca.crt - downwardAPI: items: - path: namespace fieldRef: fieldPath: metadata.namespace ``` The kubelet automatically rotates this token before it expires. An attacker who steals the token only has it for up to 1 hour. ---
Kubernetes is not a single system — it is an API server, a scheduler, a controller manager, a key-value store (etcd), a ...
RBAC is the primary mechanism for controlling who can do what in a Kubernetes cluster. Understanding it properly is esse...
Pod Security Standards replace the deprecated PodSecurityPolicy. They define three levels of security that can be enforc...
By default, Kubernetes allows all pods to communicate with all other pods in the cluster. Network Policies allow you to ...
Kubernetes Secrets are base64-encoded by default — not encrypted. Base64 is encoding, not encryption. Anyone with read a...
Service accounts are the identities that pods use to authenticate to the Kubernetes API. They are one of the most common...
Static security tools scan code and images before deployment. Falco monitors what actually happens at runtime — watching...
etcd is Kubernetes' backing store — it contains every resource definition, every secret, every configuration. Compromisi...
This lab audits a namespace for common security misconfigurations and applies fixes step by step. Part 1 — Check Current...
Aligns directly with DevOps, Site Reliability (SRE), and Platform Engineering job descriptions.