Building Django-Projects on GCPs Cloud-Build

Question

Recently, we started working to convert an established Django project from a docker stack to Google App Engine. On the way, Google Cloud Build turned out to come handy. Cloudbuild takes care of a few items in preparation of rolling out, in particular the front end part of the application.

Now when it comes to python and Django specific tasks, the obvious choice is to resort to cloudbuild as well. Therefore we tried to follow the pattern Google explains with their official NPM cloud-builder (here)

The issue we are facing is the following. When building with the official python image, the buildsteps are set up as followed:

steps:
[...]
   8 - name: 'python:3.7'                                                                                                                               
   9   entrypoint: python3                                                           
  10   args: ['-m', 'pip', 'install', '-r', 'requirements.txt']                      
  11 - name: 'python:3.7'                                                            
  12   entrypoint: python3                                                           
  13   args: ['./manage.py', 'collectstatic', '--noinput']

This works just fine for the first step, to install all requirements. GAE does that when deploying the application as well, but here it's necessary to collectstatic from the repository and installed django apps, before uploading them.

While the first step succeeds with the above, the 2nd step fails with the following error:

File "./manage.py", line 14, in <module>
) from exc
ImportError: Couldn't import Django. Are you sure it's installed and 
available on your PYTHONPATH environment variable? Did you forget to 
activate a virtual environment?

Is there a better way to approach this situation?

Answer 1

There is no need to install the requirements packages directly in the workspace folder. You can use the --user flag while installing the requirements. When using the --user it will keep the packages between the build steps.

When you install the packages directly in the folder, you will upload them with the app deploy making your upload larger then needed.

You can use something like this.

- name: 'python:3.7'
  entrypoint: python3
  args: ['-m', 'pip', 'install','-r','requirements.txt', '--user']
  id: pip_install
- name: 'python:3.7'                                                            
  entrypoint: python3                                                           
  args: ['./manage.py', 'collectstatic', '--noinput']
  waitFor:
      - pip_install

Answer 2

Anything outside the /workspace directory is not persisted between builds, so the requirements that you're installing aren't making it to the second step. From "Creating Custom Build Steps":

A custom build step runs with the source mounted under /workspace, and is run with a working directory somewhere in /workspace. Any files left in /workspace by a given build step are available to other build steps, whether those steps are run concurrently or subsequently.

One way to work around this would be to install them into the current directory instead:

- name: 'python:3.7'                                                                                                                               
  entrypoint: python3                                                               
  args: ['-m', 'pip', 'install', '-t', '.', '-r', 'requirements.txt']                      
- name: 'python:3.7'                                                            
  entrypoint: python3                                                           
  args: ['./manage.py', 'collectstatic', '--noinput']

Similarly, could also create a virtual environment and activate it for each step that requires the dependencies to be installed.