CODEWITHSUNDEEP
terraformterraform-provider-awsamazon-eks
Jaime S
Jaime S

Reputation: 1698

EKS Terraform - datasource aws_eks_cluster_auth - token expired

I've an EKS cluster deployed in AWS and I use terraform to deploy components to that cluster.

In order to get authenticated I'm using the following EKS datasources that provides the cluster API Authentication:

data "aws_eks_cluster_auth" "cluster" {
  name = var.cluster_id
}

data "aws_vpc" "eks_vpc" {
  id = var.vpc_id
}

And using the token inside several local-exec provisioners (apart of other resources) to deploy components

resource "null_resource" "deployment" {
  provisioner "local-exec" {
    working_dir = path.module
    command     = <<EOH
kubectl \
  --server="${data.aws_eks_cluster.cluster.endpoint}" \
  --certificate-authority=./ca.crt \
  --token="${data.aws_eks_cluster_auth.cluster.token}" \
  apply -f test.yaml
EOH
  }
}

The problem I have is that some resources are taking a little while to deploy and at some point when terraform executes the next resource I get this error because the token has expired:

exit status 1. Output: error: You must be logged in to the server (the server has asked for the client to provide credentials)

Is there a way to force re-creation of the data before running the local-execs?

Upvotes: 1

Views: 3216

Answers (2)

Harsha G V
Harsha G V

Reputation: 1030

By default, aws_eks_cluster_auth token is valid for ~15 minutes only. So to extend the token lifetime automatically, you can use below approach, as mentioned here

provider "kubernetes" {
  host                   = module.eks.cluster_endpoint
  cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
  exec {
    api_version = "client.authentication.k8s.io/v1alpha1"
    args        = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
    command     = "aws"
  }
}

In case of adding IAM role to args, update it as:

args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name, "--role-arn", var.iam_role_arn]

Upvotes: 2

aidanmelen
aidanmelen

Reputation: 6594

UPDATE: example moved to https://github.com/aidanmelen/terraform-kubernetes-rbac/blob/main/examples/authn_authz/main.tf

The data.aws_eks_cluster_auth.cluster_auth.token creates a token with a non-configurable 15 minute timeout.

One way to get around this is to use the sts token to create a long-lived service-account token and use that to provision the terraform-kubernetes-provider for long running kuberenetes resources.

I created a module called terraform-kubernetes-service-account to capture this common behavior of creating a service account, giving it some permissions, and output the auth information i.e. token, ca.crt, namespace.

For example:

module "terraform_admin" {
  source = "aidanmelen/service-account/kubernetes"

  name               = "terraform-admin"
  namespace          = "kube-system"
  cluster_role_name  = "terraform-admin"
  cluster_role_rules = [
    {
      api_groups     = ["*"]
      resources      = ["*"]
      resource_names = ["*"]
      verbs          = ["*"]
    },
  ]
}

provider "kubernetes" {
  alias                  = "terraform_admin_service_account"
  host                   = "https://kubernetes.docker.internal:6443"
  cluster_ca_certificate = module.terraform_admin.auth["ca.crt"]
  token                  = module.terraform_admin.auth["token"]
}

data "kubernetes_namespace_v1" "example" {
  metadata {
    name = kubernetes_namespace.ex_complete.metadata[0].name
  }
}

Upvotes: 2

Related Questions