CODEWITHSUNDEEP
kubernetes
Adam Thomason
Adam Thomason

Reputation: 1166

Prevent Kubernetes users from being able to create privileged containers

I'm currently attempting to look into whether it's possible to prevent a Kubernetes user from creating privileged containers via RBAC. I'm aware that as of Kubernetes 1.1, privileged containers are enabled by default to support underlying Docker requirements. This is fine, I'm not looking to block everybody from being able to run privileged containers.

I would, however, like to work on a principle of least privilege. For example, I'd like to prevent users being able to use something such as kubectl node-shell from being able to gain root access to a worker node, preferably through RBAC.

Is this possible?

Upvotes: 3

Views: 1769

Answers (3)

Sam Stoelinga
Sam Stoelinga

Reputation: 5021

The correct answer as of 1.23 is to use Pod Security Standards that are enforced by the Pod Security Admission controller which is included by default since 1.23. K8s has removed Pod Security Policy (PSP) in 1.25 an PSP was deprecated in 1.23.

K8s comes with 3 pre-defined Pod Security Standards: Privileged, Baseline and Restricted. Enforcing the Baseline and Restricted standards would both allow you to prevent privileged pods. However, restricted might be too restrictive for your pod so I would recommend starting with the baseline standard.

Enforcing the baseline Pod Security Standard

Enforcing a pod security standard to a namespace has the risk of preventing new pods from being deployed to the namespace. So lets do a dry-run first instead of directly enforcing baseline.

Run a dry-run and check if any warnings are thrown:

kubectl label --dry-run=server --overwrite ns default \
    pod-security.kubernetes.io/enforce=baseline

If you saw namespace/default labeled without any warnings then that means all the currently running pods inside the namespace default would have been admitted if baseline was enforced.

Assuming you had no warnings. Let's start by enforcing the baseline standard on the default namespace:

kubectl label --overwrite ns default \
    pod-security.kubernetes.io/enforce=baseline

Notice that this time the --dry-run=server parameter is not added.

Let's verify that privileged pods indeed are getting blocked.

Create a file named nginx-priv.yaml with the following content:

apiVersion: v1
kind: Pod
metadata:
  name: nginx-priv
spec:
  containers:
  - name: nginx-priv
    image: nginx:1.14.2
    ports:
    - containerPort: 80
    securityContext:
      privileged: true

Try to create the privileged pod:

kubectl apply -f nginx-priv.yaml

You should see the following output:

Error from server (Forbidden): error when creating "nginx-priv.yaml": pods "nginx-priv" is forbidden: violates PodSecurity "baseline:latest": privileged (container "nginx-priv" must not set securityContext.privileged=true)

Note you can also enforce baseline across all namespaces like this (might cause issues when deploying new pods):

kubectl label --overwrite ns --all \
  pod-security.kubernetes.io/enforce=baseline

Disclaimer: I'm the author of below source
Source: https://samos-it.com/posts/Preventing-Privileged-pods-using-Pod-Security-Admission-Standards.html

Upvotes: 0

Fabrice Jammes
Fabrice Jammes

Reputation: 3185

Once PodSecurityPolicies have been enabled on you cluster, the standard recommends creating 3 levels of policies:

  • privileged
  • baseline
  • restricted then you have to use RBAC to map these policies with your users. Here is a minimalist example for all of these: https://github.com/k8s-school/kind-travis-ci/tree/master/psp. It will at least allow to boostrap the control plane once PodSecurityPolicies are enabled, indeed kube-proxy and CNI daemonset, and coredns deployment service accounts require some rights on PodSecurityPolicies in order to be able to create their related pod.

Upvotes: 1

prometherion
prometherion

Reputation: 2299

There are several ways to achieve this and I'd like to start with the first, official one: Pod Security Policy.

The policy you want to implement is the following: Privileged

determines if any container in a pod can enable privileged mode. By default a container is not allowed to access any devices on the host, but a "privileged" container is given access to all devices on the host. This allows the container nearly all the same access as processes running on the host. This is useful for containers that want to use linux capabilities like manipulating the network stack and accessing devices.

The Kubernetes documentation provides few examples on how to interact with PSP, although a big disclaimer must be stated: the PodSecurityPolicy admission controlled must be activated on the API Server.

Besides the official admission controller, there are other projects that allow performing the denial of containers matching criteria, as well as running as privileged: Gatekeeper, Kyverno.

There are other projects that can address your use-case but wanted to share these last two since they seem the most promising ones.

Examples

Upvotes: 6

Related Questions