CODEWITHSUNDEEP
amazon-web-serviceskubernetesdevopsazure-aks
deepak
deepak

Reputation: 33

How to add taint to AWS EKS node Group?

I am having one AWS EKS Cluster up & running. I need to add node group with taint. So that I can deploy the pod on particular node group in EKS. I can do it in azure AKS using the following command.

az aks nodepool add --resource-group rg-xx --cluster-name aks-xxx --name np1 --node-count 1 --node-vm-size xxx --node-taints key=value:NoSchedule --no-wait

How to achieve same in AWS EKS?

Upvotes: 4

Views: 10691

Answers (6)

Mudit Krishna Mathur
Mudit Krishna Mathur

Reputation: 342

For those who are looking for a solution Using eksctl + yaml file

You can run the following command

eksctl create nodegroup --config-file=<clusterconfig.yaml>

<clusterconfig.yaml> is the file that you used to create your cluster. (Yes you can use the same file to create a new nodegroup/managednodegroup.) Add the new new managednodegroup and run the create command. eksctl will take a diff and create the new nodegroup. My cluster config looks like the yaml I have pasted below. I created nodegroup ng-system during the creation of the cluster. I later added a new cluster group ng-kafka, with taints, through the command I mentioned above.

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: test-cluster
  region: us-east-2
  version: "1.22"
managedNodeGroups:
  - name: ng-system
    instanceType: t3.medium
    minSize: 3
    maxSize: 10
    spot: true
    maxPodsPerNode: 35
    labels: {
      role: system
    }
  - name: ng-kafka
    instanceType: m5.large
    desiredCapacity: 2
    minSize: 1
    maxSize: 4
    spot: true
    taints:
      - {
          "key": "dedicated",
          "value": "kafka",
          "effect": "NoSchedule"
        }
    ssh:
      allow: true
    labels: {
      role: kafka
    }

Upvotes: 2

MADHAIYAN M
MADHAIYAN M

Reputation: 2068

You can Add like this in AWS:EKS:NodeGroup Cloudformation Template

Taints:
  - Effect: "NO_SCHEDULE"
    Key: !Ref NodeGroupName
    Value: "dedicated"

Upvotes: 1

Rotem jackoby
Rotem jackoby

Reputation: 22198

I haven't seen eksctl as a requirment in the question so I'm adding another 2 options.

Option 1 - AWS CLI

From here:

Kubernetes node taints can be applied to new and existing managed node groups using the AWS Management Console or through the Amazon EKS API

The following is an example of creating a node group with a taint using the AWS CLI:

aws eks create-nodegroup \
 --cli-input-json '
{
  "clusterName": "my-cluster",
  ...
  "taints": [
     {
         "key": "dedicated",
         "value": "gpuGroup",
         "effect": "NO_SCHEDULE"
     }
   ],
}'

Option 2 - Terraform

If you're using Terraform then Hashicorp AWS provider supports taints out of the box with the taint configuration block:

resource "aws_eks_node_group" "stateless-ng" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "stateless-ng"
  .
  .
  .
  
  # Block 1 
  taint {
    key = "stateless-no-schedule"
    value  = "true"
    effect = "NO_SCHEDULE"
  }

  # Block 2    
  taint {
    key = "stateless-no-execute"
    value  = "true"
    effect = "NO_EXECUTE"
  }
}

Notice that each taint needs to be configured in a sperated configuration block.

Upvotes: 1

lvthillo
lvthillo

Reputation: 30801

Kubernetes taints are supported by EKS managed NodeGroups since April 2021: https://docs.aws.amazon.com/eks/latest/userguide/node-taints-managed-node-groups.html

Upvotes: 2

afirth
afirth

Reputation: 151

if you are using eksctl with managed nodegroups, you can patch the bootstrap script to achieve what you want.

managedNodeGroups:
  - name: batch
    preBootstrapCommands:
      - sed -i '/^KUBELET_EXTRA_ARGS=/a KUBELET_EXTRA_ARGS+=" --register-with-taints=tier=batch:NoSchedule"' /etc/eks/bootstrap.sh
    tags:
      k8s.io/cluster-autoscaler/node-template/taint/tier: "batch:NoSchedule"

this relies on the code at https://github.com/awslabs/amazon-eks-ami/blob/189baaa77c14120a1b62c42bacced17ba429466b/files/bootstrap.sh#L107 not changing too much.

You can similarly enable the docker network bridge (aka --enable-docker-bridge)

    preBootstrapCommands:
      - sed -i 's/^ENABLE_DOCKER_BRIDGE=.*/ENABLE_DOCKER_BRIDGE=true/' bootstrap.sh

The tracking issue for actual EKS support is at https://github.com/aws/containers-roadmap/issues/864, and this solution comes from a comment

Upvotes: 2

pb100
pb100

Reputation: 786

You can use this example: https://eksctl.io/usage/autoscaling/#scaling-up-from-0

nodeGroups:
- name: ng1-public
  ...
  labels:
    my-cool-label: pizza
  taints:
    feaster: "true:NoSchedule"

Upvotes: 4

Related Questions