CODEWITHSUNDEEP
dockerkubernetesjenkinsgoogle-kubernetes-enginegke-networking
Raunak Gupta
Raunak Gupta

Reputation: 10799

Jenkins Docker Container can't access docker.sock in GKE Autopilot

I have installed Jenkins using helm v3 in GKE Autopilot Clustor using default chart value. I am trying to create a Docker image from Jenkins but getting permission issue (autogke-no-write-mode-hostpath).

Jenkinsfile

pipeline {
  agent {
    kubernetes {
      defaultContainer 'jnlp'
      yaml '''
        apiVersion: v1
        kind: Pod
        spec:
          containers:
          - name: nodejs
            image: node:16
            command:
            - cat
            tty: true
            resources:
              requests:
                memory: "4Gi"
                cpu: "1000m"
          - name: gcloud-sdk
            image: google/cloud-sdk:latest
            command:
            - cat
            tty: true
          - name: docker
            image: docker:latest
            command:
            - cat
            tty: true
            volumeMounts:
            - mountPath: /var/run/docker.sock
              name: docker-sock
          volumes:
            - name: docker-sock
              hostPath:
                path: /var/run/docker.sock
        '''
    }
  }
    environment {
        // GKE
        PROJECT_ID = 'example-0000'

        // Docker
        GCR_HOSTNAME = 'us.gcr.io'
        DOCKER_IMG = "${env.GCR_HOSTNAME}/${env.PROJECT_ID}/my-app"
        DOCKER_IMG_TAG = "${env.DOCKER_IMG}:${env.BRANCH_NAME}-${env.BUILD_NUMBER}"

        NODE_ENV = 'production'

        HOME = "${WORKSPACE}"
        NPM_CONFIG_CACHE = "${WORKSPACE}/.npm"
    }

    options {
        disableConcurrentBuilds(abortPrevious: true)
        parallelsAlwaysFailFast()
    }

    stages {
        stage('Build: Docker Image') {
            when {
                beforeAgent true
                anyOf { branch 'master'; branch 'sandbox' }
            }
            steps {
                container('docker') {
                    sh "sed -i 's#__NODE_ENV__#${NODE_ENV}#' ./Dockerfile"

                    sh "docker build -t ${env.DOCKER_IMG_TAG} ."

                    withCredentials([file(credentialsId: 'my-gcr-cred', variable: 'GCR_MANAGER_KEY')]) {
                        sh 'chmod 600 $GCR_MANAGER_KEY'
                        sh('cat $GCR_MANAGER_KEY | docker login -u _json_key --password-stdin https://' + "${env.GCR_HOSTNAME}")
                        sh "docker push ${DOCKER_IMG_TAG}"
                        sh "docker logout https://${env.GCR_HOSTNAME}"
                    }
                }
            }
            post {
                always {
                    sh "docker rmi ${env.DOCKER_IMG_TAG}"
                }
            }
        }
    }
}

Error that I am getting

ERROR: Unable to create pod kubernetes jenkins/pro-7-bmvlz-n2z9s-bqxmd. Failure executing: POST at: https://10.100.108.3/api/v1/namespaces/jenkins/pods. Message: admission webhook "gkepolicy.common-webhooks.networking.gke.io" denied the request: GKE Policy Controller rejected the request because it violates one or more policies: {"[denied by autogke-no-write-mode-hostpath]":["hostPath volume docker-sock in container docker is accessed in write mode; disallowed in Autopilot. Requested by user: 'system:serviceaccount:jenkins:jenkinsv2', groups: 'system:serviceaccounts,system:serviceaccounts:jenkins,system:authenticated'."]}. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook "gkepolicy.common-webhooks.networking.gke.io" denied the request: GKE Policy Controller rejected the request because it violates one or more policies: {"[denied by autogke-no-write-mode-hostpath]":["hostPath volume docker-sock in container docker is accessed in write mode; disallowed in Autopilot. Requested by user: 'system:serviceaccount:jenkins:jenkinsv2', groups: 'system:serviceaccounts,system:serviceaccounts:jenkins,system:authenticated'."]}, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=[denied by autogke-no-write-mode-hostpath], status=Failure, additionalProperties={}).

Upvotes: 0

Views: 1113

Answers (1)

Harsh Manvar
Harsh Manvar

Reputation: 30178

What you are trying to access is at the /var/run and Docker in Jenkin

However, if we see GKE auto pilot is managed K8s service and you don't have access to underlying the infrastructure.

Autopilot GKE does not allow you to use the Hostpath method or mount the folder with written permission.

You are only allowed to perform the read operation.

HostPort and hostNetwork are not permitted because node management is handled by GKE. Using hostPath volumes in write mode is prohibited while using hostPath volumes in read mode is allowed only for /var/log/ path prefixes.

Read more at : https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview

Upvotes: 1

Related Questions