CODEWITHSUNDEEP
kubernetesrancherk3s
Kyle
Kyle

Reputation: 436

How do you increase maximum pods per node in K3S?

I have tried setting max nodes per pod using the following upon install:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--max-pods 250" sh -s -

However, the K3s server will then fail to load. It appears that the --max-pods flag has been deprecated per the kubernetes docs:

--max-pods int32 Default: 110

(DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)

So with K3s, where is that kubelet config file and can/should it be set using something like the above method?

Upvotes: 3

Views: 9730

Answers (2)

dols
dols

Reputation: 753

To update your existing installation with an increased max-pods, add a kubelet config file into a k3s associated location such as /etc/rancher/k3s/kubelet.config:

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 250

edit /etc/systemd/system/k3s.service to change the k3s server args:

ExecStart=/usr/local/bin/k3s \
    server \
        '--disable' \
        'servicelb' \
        '--disable' \
        'traefik' \
        '--kubelet-arg=config=/etc/rancher/k3s/kubelet.config'

reload systemctl to pick up the service change:

sudo systemctl daemon-reload

restart k3s:

sudo systemctl restart k3s

Check the output of describe nodes with kubectl describe <node> and look for allocatable resources:

Allocatable:
  cpu:                32
  ephemeral-storage:  199789251223
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             131811756Ki
  pods:               250

and a message noting that allocatable node limit has been updated in Events:

Events:
  Type     Reason                   Age    From        Message
  ----     ------                   ----   ----        -------
  Normal   Starting                 20m    kube-proxy  Starting kube-proxy.
  Normal   Starting                 20m    kubelet     Starting kubelet.
...
  Normal   NodeNotReady             7m52s  kubelet     Node <node> status is now: NodeNotReady
  Normal   NodeAllocatableEnforced  7m50s  kubelet     Updated Node Allocatable limit across pods
  Normal   NodeReady                7m50s  kubelet     Node <node> status is now: NodeReady

Upvotes: 11

matt_j
matt_j

Reputation: 4614

As described in documentation, it is possible to set the Kubelet's configuration parameters via an on-disk config file.

NOTE: Using an on-disk config file, we can set only subset of the Kubelet's configuration parameters, that we want to override, all other Kubelet configuration values are left at their built-in defaults, unless overridden by flags.

I've created simple config file to override maxPods value (default 110):

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 250

And then we have to pass this config file as an argument during K3s installation (I recommend to specify absolute pathname):

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--kubelet-arg=config=<KUBELET_CONFIG_FILE_LOCATION>" sh -

Finally we can check that maxPods is equal to 250:

# kubectl describe nodes <NODE_NAME> | grep -i pod
  pods:               250
  pods:               250
PodCIDR:                      10.42.0.0/24
PodCIDRs:                     10.42.0.0/24

In addition you can find interesting discussion here, I believe you can find there another way to solve your issue.

Upvotes: 5

Related Questions