GCP VM Instance running a docker image with an "out of date" build argument

Question

Context

I'm attempting to build a CD pipeline in a GitHub action that creates a docker image, pushes it to Google Cloud Registry (GCR), and then restarts a VM instance with the latest image.

Problem

The VM Instance is somehow not running the latest docker image, even though the VM Instance page itself shows the right image tag, and I'm seeing the VM Instance being restarted.

The reason I suspect this is because when I SSH into the VM Instance, and run docker logs <container-id> I see logs that indicate that an old value of a Docker build argument is being used - specifically DATABASE_URL. (I've also confirmed that, after requesting endpoints, the appropriate logs show up here, so it seems like I'm viewing the right logs)

When I pull down this same image and run it locally, the DATABASE_URL is correct.

When I SSH into the VM Instance and run docker images I'm not seeing the newer images being listed. I must have broken something because this was working earlier, but I'm having trouble finding the issue here.

Question

Why isn't the VM Instance receiving the new docker images? (When running docker images, I don't see new docker images being listed).

Update

After checking the VM Instance's logs via:

sudo journalctl -u konlet-startup

I'm seeing the following error:

Error: Failed to start container: Error response from daemon: {"message":"pull access denied for gcr.io/..., repository does not exist or may require 'docker login': denied: Permission denied for \"...\" from request \"...". "}

So that explains why I'm no longer receiving new docker images.

Code Snippets

name: Continuous Delivery

on:
  push:
    branches: [master]

env:
  PROJECT_ID: ${{ secrets.GCE_PROJECT }}
  GCE_INSTANCE: ${{ secrets.GCE_INSTANCE }}
  GCE_INSTANCE_ZONE: ${{ secrets.GCE_INSTANCE_ZONE }}

jobs:
  build:
    name: Update GCP VM instance
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      # Setup gcloud CLI
      - uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
        with:
          version: "290.0.1"
          service_account_key: ${{ secrets.GCE_SA_KEY }}
          project_id: ${{ secrets.GCE_PROJECT }}

      # Configure Docker to use the gcloud command-line tool as a credential
      # helper for authentication
      - run: |-
          gcloud --quiet auth configure-docker

      # Build the Docker image
      - name: Build
        run: |-
          docker build --no-cache --build-arg DATABASE_URL=${{ secrets.DATABASE_URL }} --tag "gcr.io/$PROJECT_ID/platform:$GITHUB_SHA" .

      # Push the Docker image to Google Container Registry
      - name: Publish
        run: |-
          docker push "gcr.io/$PROJECT_ID/platform:$GITHUB_SHA"

      - name: Deploy
        run: |-
          gcloud compute instances update-container "$GCE_INSTANCE" \
            --zone "$GCE_INSTANCE_ZONE" \
            --container-image "gcr.io/$PROJECT_ID/platform:$GITHUB_SHA"

To be super sure that secrets.DATABASE_URL is actually correct, I spun up a server and sent a request within the Dockerfile:

RUN curl "http://my-url.ngrok.io/${DATABASE_URL}"

and I can confirm that it is indeed correct.