How to use terraform to execute kubectl apply in a Create a generic kubernetes object built in Terraform?

Question

Pipeline Problem

• Terraform is used to build our eks Kubernetes cluster.
• We use terraform to write a couple of config files for other services that depend on the values from terraform

Generating a generic k8s object file to be applied

• So far, I can generate the file...

locals {
  cert_issuer = {
    apiVersion = "cert-manager.io/v1"
    kind       = "ClusterIssuer"
    metadata = {
      name = "letsencrypt-prd"
    }
    spec = {
      acme = {
        # https://letsencrypt.org/docs/acme-protocol-updates/
        server = "https://acme-v02.api.letsencrypt.org/directory"

        # Email for the cert contact
        email = "contact@${var.domain}"

        # Name of a secret used to store the ACME account private key
        privateKeySecretRef = {
          name = "${var.domain}-private-key-secret"
        }

        # Zone resolvers by Route53 DNS01 challenges
        solvers = [{
          selector = {
            dnsZones = [var.domain]
          }
          dns01 = {
            route53 = {
              region = var.aws_region
              hostedZoneID = data.aws_route53_zone.domain_hosted_zone.zone_id
            }
          }
        }]
      }
    }
  }
}

resource "local_file" "cert_manager_cluster_issuer_object" {
  content  = yamlencode(local.cert_issuer)
  filename = "${path.module}/.k8s/cert-manager/cluster-issuer-letsencrypt-prd"
}

• This saves the file and I'd like to apply it using kubectl apply

Answer 1

Solution using community provider

• Found the provider "k8s" has been archived by the original authors https://github.com/ericchiang/terraform-provider-k8s
  • But it has been maintained by https://github.com/banzaicloud/terraform-provider-k8s#argument-reference

Solution for EKS

• There's no need to store the k8s object locall.
• Using eks, we can configure the connection to apply any kind of resource

provider "k8s" {
  host                   = data.aws_eks_cluster.cluster.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
  token                  = data.aws_eks_cluster_auth.cluster.token
  load_config_file       = false
}

• Then, just use the resource without creating the file

locals {
  cert_issuer = {
    apiVersion = "cert-manager.io/v1"
    kind       = "ClusterIssuer"
    metadata = {
      name = "letsencrypt-prd"
      #namespace: cert-manager it's cluster level no namespace
    }
    spec = {
      acme = {
        # https://letsencrypt.org/docs/acme-protocol-updates/
        server = "https://acme-v02.api.letsencrypt.org/directory"

        # Email for the cert contact
        email = "contact@${var.domain}"

        # Name of a secret used to store the ACME account private key
        privateKeySecretRef = {
          name = "${var.domain}-private-key-secret"
        }

        # Zone resolvers by Route53 DNS01 challenges
        solvers = [{
          selector = {
            dnsZones = [var.domain]
          }
          dns01 = {
            route53 = {
              region = var.aws_region
              # https://stackoverflow.com/questions/63402926/fetch-zone-id-of-hosted-domain-on-route53-using-terraform/63403290#63403290
              hostedZoneID = data.aws_route53_zone.domain_hosted_zone.zone_id
            }
          }
        }]
      }
    }
  }
}

# creating the cert_manager_cluster_issuer namespace
resource "k8s_manifest" "cert_manager_cluster_issuer" {
  content   = yamlencode(local.cert_issuer)
  namespace = local.cert_manager_namespace

  # depends_on = [local_file.kubeconfig]
}