AWS CodePipeline and deployingto EKS

Question

Am working on an AWS CodePipeline for building and deploying containers to EKS cluster.

It seems like AWS CodePipeline does not support a deployment action to EKS “only ECS”. I tried exploring other options like using lambda function, I found the below example for running kubectl commands in lambda

https://github.com/tmuskal/lambda-kubectl

Nonetheless, EKS uses aws-iam-authenticator in order to generate tokens for kubeconfig. Not sure how that would fit in the lambda context though.

Any advice on topic would be highly appreciated.

Answer 1

AWS doesn't support a deployment action for EKS. However, it can be achieved by using code pipeline and code build to make it continuous build and deployment for EKS cluster. Need to set up some IAM roles and permission in terms of allowing codebuild to run kubectl and deploy on eks cluster.

1. Need to create a role lets say (kubernetes_deployment) which has the permission to allow EKS to manage clusters on your behalf.

   • Permission attached to the kubernetes_deployment role

     AmazonEKSClusterPolicy

     AmazonEKSServicePolicy

     inline policy as below

     {
       "Version": "2012-10-17",
       "Statement": [
        {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": "eks:DescribeCluster",
          "Resource": "*"
        }
      ]
     }
     

2. Create a codebuild in your aws refer Refer this for buildspec

   • make sure service role used in codebuild should have sts:assume permission for kubernetes_deployment role which has access to manage eks cluster

        {
           "Version": "2012-10-17",
           "Statement": [
            {
              "Sid": "VisualEditor0",
              "Effect": "Allow",
              "Action": "sts:AssumeRole",
              "Resource": "arn:aws:iam:: 
                <accountno>:role/kubernetes_deployment"
              }
            ]
           }
     

3. Update the trust relationship for kubernetes_deployment role to allow used by codebuild service role

    {
      "Version": "2012-10-17",
      "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
             "AWS": "arn:aws:iam::<account>:role/service-role/codebuild-service-role",
              "Service": "eks.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
       }
    ]
   }
   

4. Make kubernetes_deployment role as authorized in eks cluster

   - rolearn: arn:aws:iam::<account>:role/kubernetes_deployment
     username: kubernetes_deployment
     groups:
      - system:masters
   

Answer 2

I am using a Jenkins POD to do deployments.

Pipleline script:

pipeline {
  agent {
    kubernetes {
      label 'helm-pod'
      serviceAccount 'jenkins-helm'
      containerTemplate {
        name 'helm'
        image 'wardviaene/helm-s3'
        ttyEnabled true
        command 'cat'
      }
    }
  }
  stages {
    stage('Run helm') {
      steps {
        container('helm') {
            dir('my-app'){
                git url: 'git@github.com:myrepo/my-app.git', branch: 'master', credentialsId: 'CREDENTIAL_ID'
            }
        sh '''
          HELM_BUCKET=helm-repo-dev.my-app.k8s.local
          PACKAGE=myservichart
          NAME_OVERRIDE=my-app
          ENV_OVERRIDE_FILE_PATH=my-project/helm-service-override/app-app.prod.values.yaml
          export AWS_REGION=eu-west-2

          cp -r /home/helm/.helm ~
          helm repo add metaservichart s3://${HELM_BUCKET}/charts
          IS_DEPLOYED=$(helm list |grep -E "^${NAME_OVERRIDE}" |grep DEPLOYED |wc -l| tr -s " ")
          if [ ${IS_DEPLOYED} == 0 ] ; then
            helm install --name ${NAME_OVERRIDE} -f ${VALUE_OVERRIDE_FILE_PATH} metaservichart/${PACKAGE}
          else
            helm upgrade ${NAME_OVERRIDE} -f ${VALUE_OVERRIDE_FILE_PATH} metaservichart/${PACKAGE}
          fi
          echo "deployed!"
        '''
        }
      }
    }
  }
}

CREDENTIAL_ID: Add github credentails in Jenkins and copy the id generated.

ENV_OVERRIDE_FILE_PATH: Environment specific properties file.

HELM_BUCKET: helm s3 bucket

NAME_OVERRIDE: Name to be passed to helm