Google Cloud Build - Different scopes of Dockerfile and cloudbuild.yaml

Question

I recently asked a question about why I get the error Specified public directory 'dist/browser' does not exist, can't deploy hosting to site PROJECT-ID when I´m trying to deploy to Firebase Hosting in my cloudbuild.yaml. However, since I find the question too bloated with information I tried to break it down.

I created a simple image to visualize what happens when I call gcloud builds submit --config=cloudbuild.yaml. So why can´t I access the directory dist/browser from cloudbuild.yaml even though it is processed after the Dockerfile where the directory dist/browser is created?

Answer 1

Cloud Build is best conceptualized as a series of functions (steps) applied to data in the form of a local file system (often just /workspace as this is a default volume mount added to each step, but you can add other volume mounts) and the Internet.

Output of each function (step) is self-contained unless you explicitly publish data back to one of these two sources (one of the step's volume mounts or the Internet).

In this case docker build consumes local files (not shown in your example) and generates dist/browser in the image that results but this folder is only accessible within that image; nothing is added to e.g. /workspace that you could use in subsequent steps.

In order to use that directory subsequently:

• Hack a way to mount the (file system of the) image generated by the step and extract the directory from it (not advised; possible not permitted).
• You'd need to run that image as a container and then docker cp files from it back into the Cloud Build's (VM's) file system (perhaps somewhere on /workspace).
• Not put the directory in an image in the first place (see below)

Proposal

Instead of docker build'ing an image containing the directory, deconstruct the Dockerfile into a series of Cloud Build steps. This way, the artifacts you want (if written somewhere under one of the step's volume mounts), will be available in subsequent steps:

steps:
- name: gcr.io/cloud-builders/npm
  args:
  - install
- name: gcr.io/cloud-builders/npm
  args:
  - run
  - build:ssr # Presumably this is where dist/browser is generated?
- name: firebase
  args:
  - deploy # dist/browser

NOTE Every Cloud Build step has an implicit:

- name: some-step
  volumes:
  - name: workspace
    path: /workspace

Proof

Here's a minimal Cloud Build config that uses a volume called testdir that maps to the Cloud Build VM's /testdir directory.

NOTE The example uses testdir to prove the point. Each Cloud Build step automatically mounts /workspace and this could be used instead.

The config:

• Lists the empty /testdir
• Creates a file freddie.txt in /testdir
• Lists /testdir now containing freddie.txt

options:
# volumes:
#   - name: testdir
#     path: /testdir

steps:
  - name: busybox
    volumes:
      - name: testdir
        path: /testdir
    args:
      - ash
      - -c
      - "ls -1a /testdir"
  - name: busybox
    volumes:
      - name: testdir
        path: /testdir
    args:
      - ash
      - -c
      - 'echo "Hello Freddie" > /testdir/freddie.txt'
  - name: busybox
    volumes:
      - name: testdir
        path: /testdir
    args:
      - ash
      - -c
      - "ls -1a /testdir"

NOTE Uncommenting volumes under options would remove the need to reproduce the volumes in each step.

The edited output is:

gcloud builds submit \
--config=./cloudbuild.yaml \
--project=${PROJECT}

# Lists (empty) /testdir
Starting Step #0
Step #0: Pulling image: busybox
Step #0: .
Step #0: ..

# Creates /test/freddie.txt
Starting Step #1
Step #1: Already have image: busybox
Finished Step #1

# List /testdir containing freddie.txt
Starting Step #2
Step #2: .
Step #2: ..
Step #2: freddie.txt
Finished Step #2