CODEWITHSUNDEEP
kubernetesprivilegescgroupscontainerdcpuset
PJConnol
PJConnol

Reputation: 139

/sys/fs/cgroup/cpuset.cpus Not Found After Enabling Privileged Mode In Kubernetes

I am currently running into an issue where /sys/fs/cgroup/cpuset.cpus is no longer present in my Kubernetes Pod when I enable Privileged mode. For example: I start off with a very simple pod spec (test.yaml):

apiVersion: v1
kind: Pod
metadata:
  name: test-0
spec:
  nodeSelector:
    kubernetes.io/hostname: "node1"
  containers:
    - name: pod-0
      image: ubuntu:22.04
      command: [ "/bin/sh" , "-c", "tail -f /dev/null" ]
      resources:
        requests:
          cpu: 4
          memory: 2Gi
        limits:
          cpu: 4
          memory: 2Gi

I start the pod and exec into it:

kubectl apply -f test.yaml 
kubectl exec -it test-0 -- /bin/bash

Once in the container, I run the following:

cat /sys/fs/cgroup/cpuset.cpus

The output is 44,46,100,102 i.e the cpus allocated to my pod are printed as expected using cgroupv2 filesystem

Next I remove the pod (kubectl delete pod test-0) and create a new pod with the following spec (note the new privileged securityContext):

apiVersion: v1
kind: Pod
metadata:
  name: test-0
spec:
  nodeSelector:
    kubernetes.io/hostname: "node1"
  containers:
    - name: pod-0
      image: ubuntu:22.04
      securityContext:
        allowPrivilegeEscalation: true
        privileged: true
      command: [ "/bin/sh" , "-c", "tail -f /dev/null" ]
      resources:
        requests:
          cpu: 4
          memory: 2Gi
        limits:
          cpu: 4
          memory: 2Gi

Same as before, I start the pod and exec into it:

kubectl apply -f test.yaml 
kubectl exec -it test-0 -- /bin/bash

Again, once in the container, I run the following:

cat /sys/fs/cgroup/cpuset.cpus

But this time I get the following error:

cat: /sys/fs/cgroup/cpuset.cpus: No such file or directory

Why is this happening whereby adding privileges to my container, it removes the /sys/fs/cgroup/cpuset.cpus file from my container in Kubernetes?

Kubernetes Version: 1.26.2

Cgroup driver: systemd

Containerd Version: 1.6.20

OS Version: Ubuntu 22.04.2

Kernel: 5.15.0-76-generic

Upvotes: 0

Views: 821

Answers (1)

Wenqing Xu
Wenqing Xu

Reputation: 1

With privileged being enabled, the cgroup directory of the host will be mounted into the container instead of the container's own cgroup hierarchy. if you wish to view only your container's CPU stats, you should not enable privileged for your pod

Upvotes: 0

Related Questions