CODEWITHSUNDEEP
kubernetes
xxw
xxw

Reputation: 89

Kubernetes data partition size

Submitted a bug report to Kubernetes and got no response. Checking here for any suggestions.

What happened:

The host OS is CentOS 7.6. There is about 50GB in the root partition. So I moved the docker engine data root /var/lib/docker to be under /mnt/storage, which has about 1.8TB. Kubernetes continues to use /var/lib/docker and starts to evict PODs that use anywhere close to 50GB.

What you expected to happen:

Kubernetes follows docker to use the same docker engine data root storage.

How to reproduce it (as minimally and precisely as possible):

(1) Add "data-root" to /etc/docker/daemon.json. Point to a mounted disk storage. Restart docker. (2) docker run --rm -it busybox sh -c 'while true; do df -h /; sleep 1; done'

Filesystem Size Used Available Use% Mounted on
overlay 1.8T 227.5G 1.5T 13% /

(3) Apply the following test.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  labels:
     app: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      enableServiceLinks: false
      containers:
        - name: test
          image: busybox
          imagePullPolicy: IfNotPresent
          command: ["sh","-c","while true; do df -h /mnt; sleep 1; done"]
          volumeMounts:
            - mountPath: /mnt
              name: storage
      volumes:
        - name: storage
          emptyDir: {}

The logs shows:

Filesystem Size Used Available Use% Mounted on
/dev/mapper/centos-root
50.0G 45.0G 5.0G 90% /mnt

Environment:

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:48:36Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

OS (e.g: cat /etc/os-release):

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Upvotes: 0

Views: 1441

Answers (1)

kool
kool

Reputation: 3613

In order to change default emptyDir mount path you have to use --root-dir flag in kubelet.

--root-dir string
Directory path for managing kubelet files (volume mounts,etc). (default "/var/lib/kubelet")

Simply add --root-dir=/mnt/storage to the kubelet, restart the service and it will change default mounting path.

Upvotes: 0

Related Questions