CODEWITHSUNDEEP
dockergoogle-cloud-platformyamldockerfilegoogle-cloud-build
Ranga Vittal
Ranga Vittal

Reputation: 196

How to pass substitution variables on Google Cloud build to run a dockerfile

I have set up a Github trigger on Google Cloud build. I already have a dockerfile to do the build steps. Since there is no provision to pass substitution variables I am trying to build it with cloud build configuration file (yaml) and then passing the path to the dockerfile in the configuration file.

This is the cloud build configuration file where I need to pass 5 variables for the Dockerfile to consume and the yaml file goes like this:

steps:
- name: 'gcr.io/cloud-builders/docker'
  env:
  - 'ACCESS_TOKEN=$ACCESS_TOKEN'
  - 'BRANCH=$BRANCH'
  - 'UTIL_BRANCH=$UTIL_BRANCH'
  - 'ARG_ENVIRONMENT=$ARG_ENVIRONMENT'
  - 'ARG_PYTHON_SCRIPT=$ARG_PYTHON_SCRIPT'
  args:
  - build
  - "--tag=gcr.io/$PROJECT_ID/quickstart-image"
  - "--file=./twitter/dax/processing_scripts/Dockerfile"
  - .

When the trigger runs the build, I get an error in one of the build steps in dockerfile saying that the variable is not available. It's clear that the environment variable passed in the yaml file is not being passed to the Dockerfile for consumption. And this is how the I have filled the substitution variables in on the trigger page enter image description here

Pasting the build code from step 5 where the error occurs, before which is just apt-get update commands running:

Step 5/28 : RUN git config --global url."https://${ACCESS_TOKEN}:@github.com/".insteadOf "https://github.com/" && echo $(ACCESS_TOKEN)
 ---> Running in f7b94bc2a0d9

/bin/sh: 1: ACCESS_TOKEN: not found
Removing intermediate container f7b94bc2a0d9
 ---> 30965207dcec
Step 6/28 : ARG BRANCH
 ---> Running in 93e36589ac48
Removing intermediate container 93e36589ac48
 ---> 1d1508b1c1d9
Step 7/28 : RUN git clone https://github.com/my_repo45/twitter.git -b "${BRANCH}"
 ---> Running in fbeb93dbb113
Cloning into 'twitter'...
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/my_repo45/twitter.git/'
The command '/bin/sh -c git clone https://github.com/my_repo45/twitter.git -b "${BRANCH}"' returned a non-zero code: 128
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 128```

Could anyone please point out what the issue and validate if the environment declaration is proper?

Upvotes: 1

Views: 3842

Answers (2)

Raymond Bourke
Raymond Bourke

Reputation: 76

I am not an expert in this topic and may not be doing this perfectly, but I was able to turn a substitution variable from cloud build into an environment variable in my container very simply:

First, I added the following line to my cloudbuild.yml file:

- "--build-arg=ARGUMENT_NAME=${_SUBSTITUTION_VARIABLE_NAME}"

I placed it as the second arg in the build step.

I do not have these lines in the cloudbuild.yml file:

 env:
    - '_SUBSTITUTION_VARIABLE_NAME=$_SUBSTITUTION_VARIABLE_NAME'

I don't think those lines do anything in this context. So, I only had to drop in one new line to set this up.

Then, I added this to the dockerfile:

ARG ARGUMENT_NAME
ENV ENVIRONMENT_VARIABLE_NAME=$ARGUMENT_NAME

These lines need to come after any FROM commands, according to my research.

That is all it took. I believe this is functionally identical to Cloudkollektiv's first solution, but I wanted to try to bring some more clarity as to what exactly to put in the cloudbuild and dockerfile.

Upvotes: 0

Cloudkollektiv
Cloudkollektiv

Reputation: 14699

You would still need to add the variables to the docker build command. As in your current situation, the substitution variables are only available for you within cloud build and not as a regular environment variable. A simplified example of how to pass those variables to cloud build is outlined below. Note that you can only use this with ARG in your dockerfile, not with ENV.

steps:
- name: 'gcr.io/cloud-builders/docker'
  env:
    - 'ACCESS_TOKEN=$_ACCESS_TOKEN'
  args:
    - build
    - "--build-arg=ACCESS_TOKEN=${ACCESS_TOKEN}"
    - "--tag=gcr.io/$PROJECT_ID/quickstart-image"
    - "--file=./twitter/dax/processing_scripts/Dockerfile"

Another option is to export the environment variable within the same build step your docker build command is in:

steps:
- name: 'gcr.io/cloud-builders/docker'
  entrypoint: 'bash'
  env:
    - 'ACCESS_TOKEN=$_ACCESS_TOKEN'
  args:
  - '-c'
  - |
    export ACCESS_TOKEN=${ACCESS_TOKEN}
    docker build \
      --tag=gcr.io/$PROJECT_ID/quickstart-image" \
      --file=./twitter/dax/processing_scripts/Dockerfile"

Of course, you could argue if the env field is still needed in this setup, I'll leave that up to you.

Upvotes: 3

Related Questions