Skip to main content
Cloud Security · lesson 4 of 8

Intermediate

8

Also part of Cloud Security

11
Open the Cloud Security page →
Cloud Security intermediate Lesson 4 of 8

Also part of: Kubernetes

Kubernetes Pod Security Admission: Enforcing the Restricted Profile

Label a namespace restricted and watch Kubernetes reject a privileged pod with the real violates PodSecurity message, then admit the compliant spec.

By SkillByExample · Published

cloud-security kubernetes pod-security minikube

Kubernetes ships with its own guard rail for pods. Pod Security Admission checks every new pod against the Pod Security Standards, and you turn it on per namespace with a label. No add-on and no cloud account is needed, so a local minikube cluster shows exactly what a managed cluster does.

Tested with minikube 1.38.1 (Docker driver), Kubernetes 1.35.1, kubectl 1.35.2 and busybox:1.37. A separate profile keeps the lab away from any other cluster:

$ minikube start -p psa-lab --driver=docker | tail -4
* Verifying Kubernetes components...
  - Using image gcr.io/k8s-minikube/storage-provisioner:v5
* Enabled addons: storage-provisioner, default-storageclass
* Done! kubectl is now configured to use "psa-lab" cluster and "default" namespace by default
$ kubectl --context psa-lab version | tail -1
Server Version: v1.35.1

A restricted namespace rejects a privileged pod

The namespace carries the policy as labels. enforce-version pins the rules to a release, so a cluster upgrade can’t silently tighten them:

# ns.yaml: a namespace that enforces the restricted Pod Security Standard
apiVersion: v1
kind: Namespace
metadata:
  name: shop
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: v1.35
# pod-bad.yaml: the "just make it work" pod
apiVersion: v1
kind: Pod
metadata:
  name: debug
  namespace: shop
spec:
  containers:
    - name: app
      image: busybox:1.37
      command: ["sleep", "3600"]
      securityContext:
        privileged: true
$ kubectl apply -f ns.yaml
namespace/shop created
$ kubectl apply -f pod-bad.yaml
Error from server (Forbidden): error when creating "pod-bad.yaml": pods "debug" is forbidden: violates PodSecurity "restricted:v1.35": privileged (container "app" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "app" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "app" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "app" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "app" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

The API server refused the pod before anything was scheduled, and the message is a complete to-do list. Only the first item comes from the privileged: true line. The other four are things a pod fails by saying nothing: unless the spec sets them, a container may escalate privileges, keeps the default capabilities, may run as root and runs without the default seccomp filter. Restricted requires all of them to be stated explicitly.

The compliant spec

Each item in the message maps to one field:

# pod-good.yaml: the same workload, inside the restricted profile
apiVersion: v1
kind: Pod
metadata:
  name: debug
  namespace: shop
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 65534
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: busybox:1.37
      command: ["sleep", "3600"]
      securityContext:
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]
$ kubectl apply -f pod-good.yaml
pod/debug created
$ kubectl -n shop wait --for=condition=Ready pod/debug --timeout=120s
pod/debug condition met
$ kubectl -n shop exec debug -- id
uid=65534(nobody) gid=65534(nobody) groups=65534(nobody)

Admitted and running as nobody. The pod-level securityContext sets the user and the seccomp profile for every container. Escalation and capabilities are container-level fields. Dropping privileged was a deletion. The rest were additions, which is why “restricted” feels strict: its defaults are the opposite of Kubernetes’ defaults.

Rolling it out to a namespace that already has pods

Turning on enforce in a busy namespace is risky, so preview it first. This namespace already runs a root pod:

# legacy.yaml: an existing namespace with a pod that runs as root
apiVersion: v1
kind: Namespace
metadata:
  name: legacy
---
apiVersion: v1
kind: Pod
metadata:
  name: worker
  namespace: legacy
spec:
  containers:
    - name: app
      image: busybox:1.37
      command: ["sleep", "3600"]
# worker2.yaml: a new root pod in the legacy namespace
apiVersion: v1
kind: Pod
metadata:
  name: worker2
  namespace: legacy
spec:
  containers:
    - name: app
      image: busybox:1.37
      command: ["sleep", "3600"]
$ kubectl apply -f legacy.yaml
namespace/legacy created
pod/worker created
$ kubectl label --dry-run=server --overwrite ns legacy pod-security.kubernetes.io/enforce=restricted
Warning: existing pods in namespace "legacy" violate the new PodSecurity enforce level "restricted:latest"
Warning: worker: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
namespace/legacy labeled (server dry run)
$ kubectl label --overwrite ns legacy pod-security.kubernetes.io/enforce=baseline pod-security.kubernetes.io/warn=restricted
namespace/legacy labeled
$ kubectl apply -f worker2.yaml
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "app" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "app" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "app" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "app" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
pod/worker2 created
$ kubectl -n legacy get pods
NAME      READY   STATUS    RESTARTS   AGE
worker    1/1     Running   0          3s
worker2   1/1     Running   0          1s

--dry-run=server asks the API server what would happen without changing anything: worker would be out of policy. So the rollout goes in two steps. enforce=baseline blocks the dangerous things (privileged, host namespaces) that this workload doesn’t do, and warn=restricted lets worker2 in while printing exactly what to fix. Every kubectl apply and CI deploy now shows the gap until the manifests are fixed and enforce can move to restricted. With no version label, the levels follow the cluster as restricted:latest.

Practice

  1. Set enforce=restricted on legacy for real. Are worker and worker2 evicted? Then apply worker3.yaml, a copy of worker2.yaml named worker3.
Solution 
$ kubectl label --overwrite ns legacy pod-security.kubernetes.io/enforce=restricted
Warning: existing pods in namespace "legacy" violate the new PodSecurity enforce level "restricted:latest"
Warning: worker (and 1 other pod): allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
namespace/legacy labeled
$ kubectl -n legacy get pods
NAME      READY   STATUS    RESTARTS   AGE
worker    1/1     Running   0          12s
worker2   1/1     Running   0          10s
$ kubectl apply -f worker3.yaml
Error from server (Forbidden): error when creating "worker3.yaml": pods "worker3" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "app" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "app" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "app" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "app" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

Nothing is evicted. Admission only judges pods as they are created, so the old pods keep running until something recreates them, and then they are rejected too. A Deployment’s next rollout is where an unprepared namespace breaks.

  1. Copy pod-good.yaml to pod-nouser.yaml, rename the pod debug2 and delete the runAsUser: 65534 line. Is it admitted? Does it run?
Solution 
$ kubectl apply -f pod-nouser.yaml
pod/debug2 created
$ sleep 15
$ kubectl -n shop get pod debug2
NAME     READY   STATUS                       RESTARTS   AGE
debug2   0/1     CreateContainerConfigError   0          16s
$ kubectl -n shop describe pod debug2 | grep "Error:"
  Warning  Failed     2s (x3 over 15s)  kubelet            spec.containers{app}: Error: container has runAsNonRoot and image will run as root (pod: "debug2_shop(8f152931-7b05-4017-bdf2-2518c569c500)", container: app)

Admission passes, because runAsNonRoot: true is set. The kubelet then refuses to start the container, because busybox’s default user is root. Two different checks at two different times: admission reads the spec, and the kubelet checks what the image will actually do. Either set a UID or use an image built to run as non-root.

  1. Create a namespace baseline-demo that enforces baseline, and apply a privileged pod there. Does baseline stop it?
Solution 
# priv.yaml: a privileged pod in a baseline namespace
apiVersion: v1
kind: Pod
metadata:
  name: priv
  namespace: baseline-demo
spec:
  containers:
    - name: app
      image: busybox:1.37
      command: ["sleep", "3600"]
      securityContext:
        privileged: true
$ kubectl create ns baseline-demo
namespace/baseline-demo created
$ kubectl label ns baseline-demo pod-security.kubernetes.io/enforce=baseline
namespace/baseline-demo labeled
$ kubectl apply -f priv.yaml
Error from server (Forbidden): error when creating "priv.yaml": pods "priv" is forbidden: violates PodSecurity "baseline:latest": privileged (container "app" must not set securityContext.privileged=true)

Yes. Baseline lists only the one violation that is a known escalation path. The root user and missing seccomp profile that restricted complained about are allowed under baseline, which is why baseline is a safe first step for existing workloads.

  1. Clean up the lab.
Solution 
$ minikube delete -p psa-lab
* Deleting "psa-lab" in docker ...
* Deleting container "psa-lab" ...
* Removing C:\Users\abhis\.minikube\machines\psa-lab ...
* Removed all traces of the "psa-lab" cluster.
$ kubectl config current-context
error: current-context is not set

Deleting the profile removes the cluster with every namespace and pod in it. It also unsets kubectl’s current context, so switch back to your usual cluster with kubectl config use-context <name> if you have one.

Frequently Asked Questions

What is Pod Security Admission in Kubernetes?
Pod Security Admission is a built-in admission controller, stable since Kubernetes 1.25, that checks every new pod against the Pod Security Standards. You switch it on per namespace with labels, and it can reject non-compliant pods (enforce), print a warning to the client (warn) or record an audit annotation (audit). It replaced the removed PodSecurityPolicy.
What is the difference between the baseline and restricted Pod Security Standards?
Baseline blocks known privilege escalations such as privileged containers, host namespaces and hostPath volumes, while still allowing most ordinary images to run as root. Restricted adds hardening best practice on top: the pod must run as non-root, set allowPrivilegeEscalation to false, drop all capabilities and use the RuntimeDefault or a Localhost seccomp profile.
Does enforcing a Pod Security level evict pods that are already running?
No. Pod Security Admission only checks pods when they are created or updated. Labelling a namespace restricted prints a warning listing existing pods that violate the new level, but they keep running. The level is applied the next time those pods are recreated, for example by a Deployment rollout, at which point non-compliant pods are rejected.
How do I roll out Pod Security Admission without breaking workloads?
Preview first with kubectl label --dry-run=server, which lists the existing pods that would violate the level. Then enforce a level the workloads already meet, such as baseline, and set warn and audit to restricted, so every new non-compliant pod is still admitted but reported. Fix the manifests, then raise enforce to restricted.
Why does a pod with runAsNonRoot fail with CreateContainerConfigError?
runAsNonRoot: true tells the kubelet to refuse to start a container that would run as UID 0. If the image's default user is root and the pod sets no runAsUser, admission passes, because the field is set, but the kubelet blocks the container at start. Set runAsUser to a non-zero UID or use an image whose default user is not root.