CODEWITHSUNDEEP
pythongoogle-cloud-buildgsutil
winwin
winwin

Reputation: 406

Google Cloud Build gsutil with python entrypoint

I want to use a python script to download several files from Google Storage by calling gsutil via subprocess.check_call() inside the Python script during a Cloud Build step.

Therefore, I use the following step inside the cloudbuild.yaml:

- name: 'gcr.io/cloud-builders/gsutil'
  entrypoint: 'bash'
  args: ['-c',
         'pip install google-cloud-storage && python /path_to_script/script.py --arg1 $_ARG1 --arg2 $TAG_NAME --arg3 $BRANCH_NAME --arg4 $_ARG4'
  ]

But Cloud Build throws a strange SyntaxError: invalid syntax Python error. I can run the same Python script with the same arguments locally as well as by using python:3.7-sim inside Cloud Build with that SyntaxError:

- name: 'python:3.7-slim'
  entrypoint: /bin/sh
  args: ['-c',
         'pip install google-cloud-storage && python python /path_to_script/script.py --arg1 $_ARG1 --arg2 $TAG_NAME --arg3 $BRANCH_NAME --arg4 $_ARG4'
  ]

The later example is working until gsutil is called, because it is not part of the python:3.7-slim container (therefor expected behavior).

Any helping comments on why gcr.io/cloud-builders/gsutil is throwing that error? Is the entrypoint wrongly specified?

Upvotes: 1

Views: 2781

Answers (1)

winwin
winwin

Reputation: 406

I found an easier solution by invoking the dedicated gsutil image in combination with wildcards to accomplish a similar filtering as in the python script. See simplified example below:

- name: 'gcr.io/cloud-builders/gsutil'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
    gsutil -m cp -r 'gs://bucket/*/' '/workspace'

This actually does the job for me pretty well.

Upvotes: 4

Related Questions