CODEWITHSUNDEEP
gitjenkinsjenkins-groovy
Woodsman
Woodsman

Reputation: 1179

Jenkins git checkout seems to check out more than I asked for

I have a Jenkins pipeline that checks out more than what I would have done had I just typed into the command line. I'd like to get Jenkins to check out my requested branch and only that branch, though if it wants to log changes, that's fine. I'm not sure why but when using the Jenkins plug in for Git it takes a simple git clone or checkout and turns it into many statements, and in my case, about half are unintended.

Included is the pipeline configuration snippet related to SCM, my Jenkinsfile, and the console output.

Extracted from the console log, this makes sense:

Checking out git [email protected]:SPRINT/ssc-file-generator.git into /var/jenkins_home/jobs/ssc-
cost-file-generator-dev/workspace@script to read Jenkinsfile-test
using credential cost_git

I think the following gets a commit hash code (not sure why it's needed but OK)

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository

This does some magic that I don't know why:

 > git config remote.origin.url [email protected]:SPRINT/ssc-file-generator.git # timeout=10

Fetching differences (ok, I can appreciate this)

Fetching upstream changes from [email protected]:SPRINT/ssc-file-generator.git
 > git --version # timeout=10
using GIT_SSH to set credentials Cost (dpydalsprd101.sl.cloud.mycompany.com, /home/cost/.ssh )
 > git fetch --tags --force --progress -- [email protected]:SPRINT/ssc-file-generator.git +refs/heads/*:refs/remotes/origin/* # timeout=10

I think the following fetches changes for my specific feature branch

 > git rev-parse refs/remotes/origin/feature/my_feature^{commit} # timeout=10

Somehow, for some reason, it decided to fetch info on a feature I don't have nor know where this comes from (questioning this)

 > git rev-parse refs/remotes/origin/origin/feature/my_feature^{commit} # timeout=10

Huh??

 > git config core.sparsecheckout # timeout=10

The following checks out a known and intended hashcode in my feature branch. I agree with it

 > git checkout -f ba939d0874a55bcbacf5691deafdca48a2bd7104 # timeout=10
Commit message: "Jenkinsfile update"

I don't know why it requests this (Another part of the question)

 > git rev-list --no-walk 388bc6908ad282c4896d6fa401ba4c6789bd04f6 # timeout=10

Finally, it tells me it's checking out what I really wanted.

Checking out git [email protected]:SPRINT/ssc-file-generator.git into /var/jenkins_home/jobs/ssc-cost-file-generator-dev/workspace@script to read Jenkinsfile-cloud-test
using credential cost_git

Then it really goes off the rails and does this:

 > git config remote.origin.url [email protected]:SPRINT/cost-ssc-file-generator.git # timeout=10
Fetching upstream changes from [email protected]:SPRINT/cost-ssc-file-generator.git
 > git --version # timeout=10
using GIT_SSH to set credentials cost (dpydalsprd101.sl.cloud.mycompany.com, /home/bluecost/.ssh )
 > git fetch --tags --progress [email protected]:SPRINT/cost-ssc-file-generator.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/proj-test^{commit} # timeout=10
Checking out Revision 99166f03eda38b4ed837182f0a3017da2c20d33e (refs/remotes/origin/proj-test)
Commit message: "Merge pull request #67 from SPRINT/proj-dev"
First time build. Skipping changelog.
[Pipeline] sh
 > git rev-parse refs/remotes/origin/origin/proj-test^{commit} # timeout=10
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 99166f03eda38b4ed837182f0a3017da2c20d33e # timeout=10
 > git branch -a -v --no-abbrev # timeout=10
 > git branch -D proj-test # timeout=10
 > git checkout -b proj-test 99166f03eda38b4ed837182f0a3017da2c20d33e # timeout=10

It begins with the configuration pipeline options:

SCM: Git
  Repositories
    Repostory URL: [email protected]:SPRINT/ssc-file-generator.git
    Credentials: <Jenkins credential>
  Branches to build:
    Branch Specifier (blank for 'any'): */feature/my_feature
  Repository browser: (Auto)
  Additional Behaviors: <none specified>
  Script Path: Jenkinsfile-test
  Lightweight checkout: <not checked>

Jenkinsfile-test

#!groovy
node('bcjenkins101'){
  def server = Artifactory.newServer url: 'https://na.artifactory.swg-devops.com/artifactory', credentialsId: 'cost_artifactory'
  def PASS_SLACK_CHANNEL = '#jenkins-build'
  def FAIL_SLACK_CHANNEL = '#jenkins-fail-squad1'
  def SLACK_DOMAIN_CREDENTIALS = 'slack-dom-ID'
  def CLOUD_REG = 'registry.cloud.mycompany.com'
  def CLOUD_PROJ = 'sprint-cost'
  def rtMaven = Artifactory.newMavenBuild()
  def rtDocker = Artifactory.docker server: server
  def buildInfo = Artifactory.newBuildInfo() 
  def hasReachedDockerComposeUp = false;
  def IMAGE_NAME="chq-sprint-images-docker-local.artifactory.swg-devops.com/ssc-cost-file-generator"
  def SERVICE_NAME = 'ssc-file-generator'
  def BRANCH_NAME = 'feature/migrate_to_cloud_2'

  try{
    env.SLACK_CHANNEL = PASS_SLACK_CHANNEL
    env.SLACK_DOMAIN = 'mycompany-sprint'
    env.SLACK_CREDENTIALS = SLACK_DOMAIN_CREDENTIALS
    env.CHANGE_LIST = 'true'
    env.TEST_SUMMARY = 'true'
    env.BRANCH_NAME= 'proj-test'
    env.NOTIFY_SUCCESS = 'true'
    env.DISPLAY = 'server.sl.bluecloud.mycompany.com:1'
    env.CLOUD_REGISTRY = "${CLOUD_REG}/${CLOUD_PROJ}"

    stage('Clone Git'){
        git url: '[email protected]:SPRINT/ssc-file-generator.git', branch: "${env.BRANCH_NAME}", credentialsId: 'cost_git'
        sh 'ls -la'
    }

    stage ('Artifactory config') {
        rtMaven.tool = 'Maven 3.5.3'
        rtMaven.deployer releaseRepo: 'sprint-libs-release-maven-local', snapshotRepo: 'sprint-libs-release-maven-local', server: server
        rtMaven.resolver releaseRepo: 'sprint-cost-libs-maven-virtual', snapshotRepo: 'sprint-bluecost-libs-maven-virtual', server: server
        buildInfo = Artifactory.newBuildInfo()
    }

    stage ('Build/Test'){
         rtMaven.run pom: 'pom.xml', goals: 'clean package', buildInfo: buildInfo
         junit(testResults: 'backend/target/surefire-reports/*xml', allowEmptyResults: true)
    }

    stage ('Exec Maven') {
        rtMaven.run pom: 'pom.xml', goals: '-Dproject.build.sourceEncoding=UTF-8 -Dproject.reporting.outputEncoding=UTF-8 clean package -DskipTests -Dskip.surefire.tests=true -Dskip.failsafe.tests=true', buildInfo: buildInfo
        junit(testResults: 'target/surefire-reports/*.xml', allowEmptyResults: true)
    }

    stage("Build, Tag and Push File-Generator Dockerfile"){
        tagDockerApp = "${env.CLOUD_REGISTRY}/${SERVICE_NAME}:${BUILD_NUMBER}"
        echo "Docker build image"
        def dockerfile = 'Dockerfile-cloud'
        echo "Docker push for the BUILD_NUMBER: ${BUILD_NUMBER}" + " push to " + "${CLOUD_REG}/${CLOUD_PROJ}"
        docker.withRegistry('https://registry.cloud.mycompany.com', 'cost_byoi_rules-service_pushbot') {
            def cloudImage = docker.build(tagDockerApp, "-f ${dockerfile} ./")
            cloudImage.push()
        }
    }    

    stage('Build & push docker images')
    {
        tagDockerApp = "${env.CLOUD_REGISTRY}/${SERVICE_NAME}:${BUILD_NUMBER}"
        echo "Docker build image"
        def dockerfile = 'Dockerfile-cloud'
        // docker.build(tagDockerApp, "-f ${dockerfile} ./")
        echo "Docker push for the BUILD_NUMBER: ${BUILD_NUMBER}" + " push to " + "${CLOUD_REG}/${CLOUD_PROJ}"
        docker.withRegistry('https://registry.cloud.mycompany.com', 'cost_byoi_rules-service_pushbot') {
            def cloudImage = docker.build(tagDockerApp, "-f ${dockerfile} ./")
            cloudImage.push()
        }
    }

    stage('Set image tag to :approved'){
        hasReachedDockerComposeUp=false;
        REPO = "chq-sprint-images-docker-local"
        sh "docker tag ${IMAGE_NAME}:test ${IMAGE_NAME}:approved"
        buildInfo = rtDocker.push("${IMAGE_NAME}:approved", REPO , buildInfo)
        server.publishBuildInfo buildInfo
    }


  }catch(error){
      currentBuild.result ='FAILURE'
      env.SLACK_CHANNEL = FAIL_SLACK_CHANNEL
      throw error
  } finally {
      sh '''
      docker image prune -f
      docker volume prune -f
      '''
      // notifier.notifyResult()
  }

}

Jenkins console output

Started by user <my_email>
Checking out git [email protected]:SPRINT/ssc-file-generator.git into /var/jenkins_home/jobs/ssc-cost-file-generator-dev/workspace@script to read Jenkinsfile-test
using credential cost_git
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url [email protected]:SPRINT/ssc-file-generator.git # timeout=10
Fetching upstream changes from [email protected]:SPRINT/ssc-file-generator.git
 > git --version # timeout=10
using GIT_SSH to set credentials Cost (dpydalsprd101.sl.cloud.mycompany.com, /home/cost/.ssh )
 > git fetch --tags --force --progress -- [email protected]:SPRINT/ssc-file-generator.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/feature/my_feature^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/feature/my_feature^{commit} # timeout=10
Checking out Revision ba939d0874a55bcbacf5691deafdca48a2bd7104 (refs/remotes/origin/feature/my_feature)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ba939d0874a55bcbacf5691deafdca48a2bd7104 # timeout=10
Commit message: "Jenkinsfile update"
 > git rev-list --no-walk 388bc6908ad282c4896d6fa401ba4c6789bd04f6 # timeout=10
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on bcjenkins_slave_dpydalsprd101 in /home/bcjenkins/Taas_Jenkins/workspace/ssc-cost-file-generator-dev
[Pipeline] {
[Pipeline] newArtifactoryServer
[Pipeline] newMavenBuild
[Pipeline] newBuildInfo
[Pipeline] stage
[Pipeline] { (Clone Git)
[Pipeline] git
using credential cost_git
Fetching changes from the remote Git repository
 > git rev-parse --is-inside-work-tree # timeout=10
 > git config remote.origin.url [email protected]:SPRINT/ssc-file-generator.git # timeout=10
Fetching upstream changes from [email protected]:SPRINT/ssc-file-generator.git
 > git --version # timeout=10
using GIT_SSH to set credentials Cost (dpydalsprd101.sl.cloud.mycompany.com, /home/cost/.ssh )
 > git fetch --tags --progress [email protected]:SPRINT/ssc-file-generator.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse refs/remotes/origin/proj-test^{commit} # timeout=10
Checking out Revision 99166f03eda38b4ed837182f0a3017da2c20d33e (refs/remotes/origin/proj-test)
Commit message: "Merge pull request #67 from SPRINT/proj-dev"
First time build. Skipping changelog.
[Pipeline] sh
 > git rev-parse refs/remotes/origin/origin/proj-test^{commit} # timeout=10
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 99166f03eda38b4ed837182f0a3017da2c20d33e # timeout=10
 > git branch -a -v --no-abbrev # timeout=10
 > git branch -D proj-test # timeout=10
 > git checkout -b proj-test 99166f03eda38b4ed837182f0a3017da2c20d33e # timeout=10
+ ls -la
<my file list in workspace. Here I can see it doesn't have my version>
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Artifactory config)
[Pipeline] newBuildInfo
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build/Test)
[Pipeline] artifactoryMavenBuild
Jenkins Artifactory Plugin version: 3.6.2
Artifactory integration is enabled

Upvotes: 2

Views: 3330

Answers (1)

apr_1985
apr_1985

Reputation: 1962

This is correct behavior. The first chunk of git commands ending at

Running in Durability level: MAX_SURVIVABILITY

is the Jenkins Master doing a lightweight (sparsecheckout) checkout of the branch to get the Jenkinsfile from the repo so it know what agent to allocate and what the pipeline is. This is evident in the log message

Checking out git [email protected]:SPRINT/ssc-file-generator.git into /var/jenkins_home/jobs/ssc-cost-file-generator-dev/workspace@script to read Jenkinsfile-cloud-test

using credential cost_git

specifically the bit to read Jenkinsfile-cloud-test

the git rev-list --no-walk 388bc6908ad282c4896d6fa401ba4c6789bd04f6 bit is so that Jenkins can build a list of files that have changed since the last build in case your pipeline has any changeSet conditions in it.

You will then see that you Pipeline allocates an executor in your case bcjenkins_slave_dpydalsprd101

The Pipeline then starts and THEN your pipeline command is run

git url: '[email protected]:SPRINT/ssc-file-generator.git', branch: "${env.BRANCH_NAME}", credentialsId: 'cost_git'

and the checkout happens on the executor node starting at [Pipeline] git

So essentially the first bit of Git stuff before the pipeline starts you will see in any Jenkins job that uses a Jenkinsfile as it is Jenkins loading the correct file up.

Upvotes: 1

Related Questions