CODEWITHSUNDEEP
pythondockergit-submodulesgoogle-cloud-build
Connor
Connor

Reputation: 423

GCP Cloudbuild Git Submodule Python Installation Failing in Docker Build

I'm trying to use GCP Cloudbuild to deploy a python project managed with UV. The project has a private git submodule that I'm cloning with an ssh setup in cloudbuild yaml (taken from this example)

The build seemingly clones the repo fine, installs the top level dependencies and then fails on uv pip install -e ./lib/db-client.

...
Step #1: Submodule 'lib/db-client' ([email protected]:path/db-client.git) registered for path 'lib/db-client'
Step #1: Cloning into '/workspace/Service/lib/db-client'...
Step #1: Submodule path 'lib/db-client': checked out '28f72f4...3443'
... goes on to install the dependencies ...

error: /app/lib/db-client does not appear to be a Python project, as neither `pyproject.toml` nor `setup.py` are present in the directory

I have successfully run my service locally from scratch with:

git clone [email protected]:path/service.git
cd service
uv sync
uv pip install -e ./lib/db-client
uv run uvicorn app:app --host 0.0.0.0 --port 8080 --reload

My cloudbuild is the following

 steps:
  - name: 'gcr.io/cloud-builders/git'
    secretEnv: ['SSH_KEY']
    entrypoint: 'bash'
    args:
    - -c
    - |
      echo "$$SSH_KEY" >> /root/.ssh/id_rsa
      chmod 400 /root/.ssh/id_rsa
      cp known_hosts.github /root/.ssh/known_hosts
    volumes:
    - name: 'ssh'
      path: /root/.ssh

  # Clone the repository
  - name: 'gcr.io/cloud-builders/git'
    args:
    - clone
    - --recurse-submodules
    - [email protected]:path/service.git
    volumes:
    - name: 'ssh'
      path: /root/.ssh

  # Initialize and update submodules
  - name: 'gcr.io/cloud-builders/git'
    entrypoint: 'bash'
    args:
    - -c
    - |
      cd EmailService
      git submodule init
      git submodule update --init --recursive
    volumes:
    - name: 'ssh'
      path: /root/.ssh

  # Build the container image
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/service', '.']

  # Push the container image to Container Registry
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'gcr.io/$PROJECT_ID/service']

  # Deploy container image to Cloud Run
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args: ['run', 'deploy', 'service', '--image', 'gcr.io/$PROJECT_ID/service', '--region', 'us-central1']

images:
  - gcr.io/$PROJECT_ID/service

availableSecrets:
  secretManager:
    - versionName: projects/$PROJECT_ID/secrets/github-ssh-key/versions/latest
      env: 'SSH_KEY'

options:
  logging: CLOUD_LOGGING_ONLY

Any help on why it's not correctly handling the submodule would be super helpful.

Upvotes: 0

Views: 39

Answers (1)

Connor
Connor

Reputation: 423

Resolved in large part to this comment.

TLDR updated the git portion of cloudbuild.yaml to:

  - name: 'gcr.io/cloud-builders/git'
    secretEnv: ['SSH_KEY']
    entrypoint: 'bash'
    args:
    - -c
    - |
      echo "$$SSH_KEY" >> /root/.ssh/id_rsa
      chmod 400 /root/.ssh/id_rsa
      cp known_hosts.github /root/.ssh/known_hosts
    volumes:
    - name: 'ssh'
      path: /root/.ssh

  - name: 'gcr.io/cloud-builders/git'
    entrypoint: 'bash'
    args:
    - '-c'
    - |
        git submodule init
        git submodule update
    volumes:
    - name: 'ssh'
      path: /root/.ssh

Upvotes: 0

Related Questions