Check if an AWS S3 Bucket exists else create it

Question

I am trying to use a Jenkinsfile to create an Amazon S3 Bucket for a project, however I want to first check if the bucket exists, else create the bucket with the parameter given.

I am implementing this in the Create Bucket stage of the Jenkinsfile using this command:

script {
  if ("aws s3api bucket-exists --bucket ${params.BUCKET}") { // Check if the bucket exists
     echo 'Bucket already exists'
  } 
  else ("aws s3api create-bucket --bucket ${params.BUCKET} --region ${params.REGION} --create-bucket-configuration LocationConstraint=${params.REGION}") { // Create the bucket if it does not exist
     echo 'Bucket created'
  }
}

Here's my Jenkinsfile:

pipeline {  
  agent any
  parameters {
    string(name: 'BUCKET', defaultValue: 's3-pipeline', description: 'Amazon S3 Bucket Name')
    string(name: 'PREFIX', defaultValue: 'my-website', description: 'Amazon S3 Bucket Application Directory')
    string(name: 'REGION', defaultValue: 'eu-west-1', description: 'Amazon S3 Bucket Region')
    string(name: 'BUILD', defaultValue: 'public/', description: 'Application Build Directory')
    string(name: 'INDEX', defaultValue: 'index.html', description: 'Application Index File')
  }  
  stages {  
    stage('Build project') {  
      steps {
        echo 'Running build project phase'
        sh 'npm install' // Install packages
        sh 'npm run clean' // Clean project 
        sh 'npm run build' // Build project 
        sh 'ls' // List project files
      }  
    //}
    stage('Create Bucket') {  
      steps {  
        echo 'Running create bucket phase'
        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'AWSCredentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) { 
          script {
            if ("aws s3api bucket-exists --bucket ${params.BUCKET}") { // Check if the bucket exists
              echo 'Bucket already exists'
            } 
            else ("aws s3api create-bucket --bucket ${params.BUCKET} --region ${params.REGION} --create-bucket-configuration LocationConstraint=${params.REGION}") { // Create the bucket if it does not exist
              echo 'Bucket created'
            }
          }
        }  
      }   
    }  
    stage('Deploy') {  
      steps {  
        echo 'Running deploy phase'
        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'AWSCredentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) { 
          sh 'aws s3 ls' // List AWS S3 buckets
          sh "aws s3 sync ${params.BUILD} s3://${params.BUCKET}/${params.PREFIX} --delete" // Sync project files with AWS S3 Bucket project path
        }  
      }   
    }
  }
  post { 
    success {  
      echo "Deployment to Amazon S3 suceeded for JOB - ${env.JOB_NAME} with BUILD NUMBER - ${env.BUILD_NUMBER}"
      echo "The Jenkins BUILD URL of this project is: ${env.BUILD_URL}" 
      echo "The Amazon S3 BUILD URL of this project is: https://${params.BUCKET}.s3-${params.REGION}.amazonaws.com/${params.PREFIX}/${params.INDEX}" 
      // mail to: team@example.com, subject: "Deployment to Amazon S3 suceeded for JOB - ${env.JOB_NAME} with BUILD NUMBER - ${env.BUILD_NUMBER}"
    }  
    failure {  
      echo "Deployment to Amazon S3 failed for JOB - ${env.JOB_NAME} with BUILD NUMBER - ${env.BUILD_NUMBER}"
      echo "The Jenkins BUILD URL of this project is: ${env.BUILD_URL}"
      // mail to: team@example.com, subject: "Deployment to Amazon S3 failed for JOB - ${env.JOB_NAME} with BUILD NUMBER - ${env.BUILD_NUMBER}"
    }  
  }  
}

However, it was returning to me that the bucket already exists, even though it's not there.

How to check to see if the bucket exists and then make as a decision based on a return value to create the bucket if it does not exist.

The AWS S3 CLI bucket-exists method documentation does not see to provide that for me, rather they just have this:

Wait until 200 response is received when polling with head-bucket. It will poll every 5 seconds until a successful state has been reached. This will exit with a return code of 255 after 20 failed checks.

How can I go about this?

Answer 1

The following code is Groovy code :

script {
  if ("aws s3api head-bucket --bucket ${params.BUCKET}") { // Check if the bucket exists
     echo 'Bucket already exists'
  } 
  else ("aws s3api create-bucket --bucket ${params.BUCKET} --region ${params.REGION} --create-bucket-configuration LocationConstraint=${params.REGION}") { // Create the bucket if it does not exist
     echo 'Bucket created'
  }
}

In Groovy, the condition if ("some random string") { ... } will always be evaluated to true, unless the given string is evaluated as falsy (see Groovy documentation for more details).

And more importantly, the given command is not executed! It's just a string evaluated to true.

If you want to execute a shell command in a Jenkins pipeline, you need to use the step sh, which is NOT what you're doing here.

You could try this :

script {
  def status = sh(script: "aws s3api head-bucket --bucket ${params.BUCKET}", returnStatus: true)
  if (status == 0) {
    echo 'Bucket already exists'
  } else {
    echo 'Bucket created'
  }
}

Or, more specifically :

script {
  def status = sh(script: "aws s3api head-bucket --bucket ${params.BUCKET}", returnStatus: true)
  if (status != 0) {
    sh "aws s3api create-bucket --bucket ${params.BUCKET} --region ${params.REGION} --create-bucket-configuration LocationConstraint=${params.REGION}"
  }
}

To implement try and catch, you could do this:

stage('Create Bucket') {  
  steps {  
    echo 'Running create bucket phase'
    withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'AWSCredentials', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) { 
      script {
        def status = sh(script: "aws s3api head-bucket --bucket ${params.BUCKET}", returnStatus: true)
        def create = sh(script: "aws s3api create-bucket --bucket ${params.BUCKET} --region ${params.REGION} --create-bucket-configuration LocationConstraint=${params.REGION}", returnStatus: true)
        try {
          if (status == 0) { // Check if the bucket exists
            echo 'Bucket already exists'
          } 
          else create() {  // Create the bucket if it does not exist
            echo 'Bucket created'
          }
        } catch (err) {
          echo "Caught: ${err}"
          currentBuild.result = 'SUCCESS'
        }
      }
    }  
  }   
}