Reputation: 2843
I'm following the example for creating an EKS managed node group from https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html
The configuration requires me to pass additional arguments to the /etc/eks/bootstrap.sh
script via the --kubelet-extra-args
argument.
My EKS worker nodes are configured via a Terraform resource aws_eks_node_group
I can't find any option for configuring the resource that would allow me to pass the --kubelet-extra-args
arguments.
Am I looking at the wrong place or is there no way to achieve this?
Upvotes: 2
Views: 11099
Reputation: 1
You can directly pass this in your node group config :
1mg-stag-mng-prom-OD-v1" = {
"desired_capacity" = 1
"max_capacity" = 2
"min_capacity" = 1
"instance_types" = ["r6a.xlarge"]
"capacity_type" = "ON_DEMAND"
"name" = "1mg-stag-mng-prom-OD-v1"
"kubelet_extra_args" = "--cluster-dns=169.254.20.25"
}
Upvotes: 0
Reputation: 2776
--kubelet-extra args
needs to be passed to the bootstrap.sh
script. When ami_type
is not "CUSTOM", EKS generates the bootstrap command and appends it at the end of the user-data, which makes it hard to customise there.
It can be done without a custom AMI (but with a launch template) with this in the user-data script. (see this answer for more details on the user-data script)
# We add another variable the we control to pass even more stuff to kubelet
sed -i '/KUBELET_EXTRA_ARGS$/{s|$| \\|;a\ \ \ \ $KUBELET_CUSTOM_ARGS'$'\n''}' /etc/systemd/system/kubelet.service /etc/eks/containerd/kubelet-containerd.service
cat > /etc/systemd/system/kubelet.service.d/40-kubelet-custom-args.conf <<'EOF'
[Service]
Environment='KUBELET_CUSTOM_ARGS=--allowed-unsafe-sysctls "net.core.somaxconn"'
EOF
This uses sed on the unit file to another variable that is passed to kubelet KUBELET_CUSTOM_ARGS
(since EKS fills in several things, like the max-pods in KUBELET_EXTRA_ARGS
and I don't want to override those)
The actual parameters that you want to pass to kubelet will likely differ.
I see the recommendation against calling the bootstrap.sh from the user-data when not using a custom AMI has been removed - it would likely call it before the AWS generated command, so that would work as well, I prefer to let EKS control the parameters.
Upvotes: 2
Reputation: 789
If you need to pass the --kubelet-extra-args
you have the option of passing user data to the instance that can be used to perform common automated configuration tasks and even run scripts after the instance starts.
In my point of view, you should have a tpl (template) file with the script you will need to run when the node is created.
userdata.tpl file like this
#!/bin/bash
%{ if length(kubelet_extra_args) > 0 }
export KUBELET_EXTRA_ARGS="${kubelet_extra_args}"
%{ endif }
%{ if length(kubelet_extra_args) > 0 || length (bootstrap_extra_args) > 0 || length (after_cluster_joining_userdata) > 0 }
/etc/eks/bootstrap.sh --apiserver-endpoint '${cluster_endpoint}' --b64-cluster-ca '${certificate_authority_data}' ${bootstrap_extra_args} '${cluster_name}'
The previous userdata.tpl file was called using a templatefile function that renders all the values on the script.
In another file you gonna have, for instance, a resource called aws_launch_template
or aws_launch_configuration
that includes an user_data base64encode
input like this.
Finally, apply all the changes and then create new nodes, they will be created with the new configuration.
Complete EKS node groups implementation here and an example of how to deploy it here
I hope it may useful for you.
Upvotes: 5