CODEWITHSUNDEEP
jenkinsjenkins-pipelineterraform
George Richardson
George Richardson

Reputation: 1268

Terraform cannot pull modules as part of jenkins pipeline

I have a jenkinsfile that was working and able to deploy some infrastructure automatically with terraform. Unfortunately after adding a terraform module with a git source it stopped working with the following error:

+ terraform init -input=false -upgrade

Upgrading modules...

- module.logstash

  Updating source "git::https://bitbucket.org/*****"

Error downloading modules: Error loading modules: error downloading 'https://bitbucket.org/*****': /usr/bin/git exited with 128: Cloning into '.terraform/modules/34024e811e7ce0e58ceae615c545a1f8'...

fatal: could not read Username for 'https://bitbucket.org': No such device or address



script returned exit code 1

The urls above were obfuscated after the fact. Below is the cut down module syntax:

module "logstash" {
  source             = "git::https://bitbucket.org/******"
  ...
}

Below is the Jenkinsfile:

pipeline {
  agent {
    label 'linux'
  }
  triggers {
    pollSCM('*/5 * * * *')
  }
  stages {
    stage ('init') {
      steps {
        sh 'terraform init -input=false -upgrade'
      }
    }
    stage('validate') {
      steps {
        sh 'terraform validate -var-file="production.tfvars"'
      }
    }
    stage('deploy') {
      when {
        branch 'master'
      }
      steps {
        sh 'terraform apply -auto-approve -input=false -var-file=production.tfvars'
      }
    }
  }
}

I believe this to be a problem with terraform internally using git to checkout the module but Jenkins has not configured the git client within the pipeline job itself. Preferably I would be able to somehow pass the credentials used by the multibranch pipeline job into the job itself and configure git but I am at a loss of how to do that. Any help would be appreciated.

Upvotes: 1

Views: 3338

Answers (1)

George Richardson
George Richardson

Reputation: 1268

So I found a non-ideal solution that requires you to specify the credentials inside your Jenkinsfile rather than automatically using the credentials used by the job for checkout. 

withCredentials([usernamePassword(credentialsId: 'bitbucketcreds', passwordVariable: 'GIT_PASS', usernameVariable: 'GIT_USER')]) {
  sh "git config --global credential.helper '!f() { sleep 1; echo \"username=${env.GIT_USER}\\npassword=${env.GIT_PASS}\"; }; f'"
  sh 'terraform init -input=false -upgrade'
  sh 'git config --global --remove-section credential'
}

The trick is to load the credentials into environment variables using the withCredentials block and then I used the answer from this question to set the credential helper for git to read in those creds. You can then run terraform init and it will pull down your modules. Finally it clears the modified git settings to hopefully avoid contaminating other builds. Note that the --global configuration here is probably not a good idea for most people but was required for me due to a quirk in our Jenkins agents.

If anyone has a smoother way of doing this I would be very interested in hearing it.

Upvotes: 2

Related Questions