Python Multistage Docker build with setup.py

Question

I am trying to rewrite my Python Docker build to a multistage build. I have been mimicking this guide in order to do so.

Here is my Dockerfile:

FROM python:3.8-slim AS compile

RUN apt-get upgrade && apt-get update
RUN apt-get install -y --no-install-recommends build-essential gcc

RUN python -m venv /opt/venv
ENV PATH='/opt/venv/bin:$PATH'

COPY requirements.txt .

RUN pip install --upgrade pip \
    && pip install -r requirements.txt \
    && pip install gunicorn

ADD src/ .
ADD setup.py .
RUN pip install .

FROM python:3.8-slim AS build

ENV GROUP_ID=1000 \
    USER_ID=1000

ENV PYTHONUNBUFFERED=1 \
    PYTHONIOENCODING=UTF-8

COPY --from=compile /opt/venv /opt/venv
ENV PATH ='opt/venv/bin:$PATH'

RUN addgroup -g $GROUP_ID WWW
RUN adduser -D -u $USER_ID -G WWW WWW -s /bin/sh && chown WWW:WWW -R /var/www

USER WWW

EXPOSE 5000

ENTRYPOINT ["gunicorn"]
CMD ["-w", "4", "--bind", "0.0.0.0:5000", "src:wsgi"]

Here is the setup.py. It currently holds just random data from this tutorial, and looks like this. I suspect the error is at packages=setuptools.find_packages(), though I cannot find out how to fix it.

import setuptools

setuptools.setup(
    name="example-pkg-YOUR-USERNAME-HERE", # Replace with your own username
    version="0.0.1",
    author="Example Author",
    author_email="author@example.com",
    description="A small example package",
    long_description="long_description",
    long_description_content_type="text/markdown",
    url="https://github.com/pypa/sampleproject",
    packages=setuptools.find_packages(),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.6',
)

On Step 11/24, when runnint pip install . the compilation halts with only Processing / as information. This is probably due to my setup script, however I have not found a way to fix it.

Step 9/24 : ADD src/ .
 ---> d51425cd9672
Step 10/24 : ADD setup.py .
 ---> 2d0378201b2a
Step 11/24 : RUN pip install .
 ---> Running in a747453f98a5
Processing /

Here is the file structure of my project

❯ tree
.
|-- Dockerfile
|-- bin
|   |-- activate
|   |-- activate.csh
|   |-- activate.fish
|   |-- easy_install
|   |-- easy_install-3.7
|   |-- pip
|   |-- pip3
|   |-- pip3.7
|   |-- python -> python3
|   `-- python3 -> /usr/local/bin/python3
|-- foo.log
|-- include
|-- lib
|   `-- python3.7
|       `-- site-packages
            ...
            lots of files...
            ...

|-- pyvenv.cfg
|-- requirements.txt
|-- setup.py
`-- src
    |-- app.py
    `-- wsgi.py

I have had the build working as a singe stage build, bu then without the src folder, and app.py and wsgi.py directly inside project folder.

Any help is appreciated!

----------UPDATE---------

I have tried running the pip install . locally in another location with only src/ folder and setup.py with the following result:

Here is the output:

Processing /Users/xxx/xxx/test
Building wheels for collected packages: example-pkg-YOUR-USERNAME-HERE
  Building wheel for example-pkg-YOUR-USERNAME-HERE (setup.py) ... done
  Created wheel for example-pkg-YOUR-USERNAME-HERE: filename=example_pkg_YOUR_USERNAME_HERE-0.0.1-cp37-none-any.whl size=1391 sha256=0d2d6eca20042c47f1cbe6a022949e2770f167ae6324a8c2f8ff473baaac7fa7
  Stored in directory: /private/var/folders/7v/5w20zhw90g31z2cl95qngb_80000gn/T/pip-ephem-wheel-cache-e_jn3t7n/wheels/50/63/3f/b109a0dbe8dbf853eb63d219a39f3e501c43d52305bc1c5b38
Successfully built example-pkg-YOUR-USERNAME-HERE
Installing collected packages: example-pkg-YOUR-USERNAME-HERE
Successfully installed example-pkg-YOUR-USERNAME-HERE-0.0.1

Answer 1

You need to change the working directory, as you are copying all your code to the root dir. When you then run pip install . from /, pip copies the entire filesystem to a temporary directory in order to install from there. I wonder what happens if you wait for it to finish?

Also, I think that one of the earlier lines might be wrong. I usually do apt update && apt -y upgrade and not the other way around.

Try this:

FROM python:3.8-slim AS compile

RUN apt-get update && apt-get -y upgrade
RUN apt-get install -y --no-install-recommends build-essential gcc

RUN python -m venv /opt/venv
ENV PATH='/opt/venv/bin:$PATH'

WORKDIR /usr/src/app
COPY requirements.txt .

RUN pip install --upgrade pip \
    && pip install -r requirements.txt \
    && pip install gunicorn

# To match the directory structure at host:
ADD src/ ./src
ADD setup.py .
RUN pip install .

For more information on how to write dockerfiles for the Python images, see here.