Setting a non-root user in a Dockerfile and installing pip packages

Question

I have this multi-stage build:

FROM mcr.microsoft.com/azure-functions/python:3.0-python3.8 as intermediate

RUN apt-get update && \
    apt-get install -y apt-utils && apt-get install -y git && \
    wget https://packages.microsoft.com/config/ubuntu/20.10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg -i packages-microsoft-prod.deb && \
    apt-get update; \
    apt-get install -y apt-transport-https && \
    apt-get update && \
    apt-get install -y dotnet-sdk-5.0

ARG SPECKLE_ENFORCE_SSL
ENV SPECKLE_ENFORCE_SSL=false
ARG ARTIFACTS_KEYRING_NONINTERACTIVE_MODE
ENV ARTIFACTS_KEYRING_NONINTERACTIVE_MODE=true

ARG AZ_DEVOPS_TOKEN
ENV AZ_DEVOPS_TOKEN=$AZ_DEVOPS_TOKEN
ENV PYTHONUNBUFFERED 1


RUN pip install --upgrade pip --no-cache-dir && \
    pip install pyyaml numpy lxml artifacts-keyring pytest --no-cache-dir && \
    pip install -i https://$AZ_DEVOPS_TOKEN@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

# Ditch the intermediate layer
# IN THIS FINAL STAGE I WANT TO ADD AN USER AND USE IT

FROM mcr.microsoft.com/azure-functions/python:3.0-python3.8

ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
  AzureFunctionsJobHost__Logging__Console__IsEnabled=true

COPY --from=intermediate /usr/local/lib/python3.8/site-packages/ /usr/local/lib/python3.8/site-packages/
ADD requirements.txt /

# SO I DID THE FOLLOWING: 
RUN addgroup --system --gid 1250 appgroup \
&& adduser --system -uid 1250 --ingroup appgroup --shell /bin/sh appuser && \
echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
RUN chown -R 1250:1250 /home/ && chmod -R 0700 /home/
USER appuser

RUN pip install -r /requirements.txt

ADD . /home/appuser/wwwroot
WORKDIR /home/appuser/wwwroot/HttpUploadTrigger/
#RUN export PATH=/home/.local/bin:$PATH

# I  CANNOT INSTALL pytest package 
RUN pip install pytest && pytest --verbose

At the second stage, I added the appgroup group and the appuser user which belong to addgroup group

And it works I am using appuser to run the container when I remove the RUN pip install pytest && pytest --verbose line above, then I got this:

> docker exec -ti af-fem-uploader bash
appuser@fdf737a9e6ae:~/appuser/wwwroot/HttpUploadTrigger$

But when I add the RUN pip install pytest && pytest --verbose step, my output on the build is:

> CACHED [stage-1 4/9] RUN addgroup --system --gid 1250 appgroup && adduser --system -uid 1250 --ingroup appgroup --shell /bin/sh appuser && echo '%sudo ALL=  0.0s
 => CACHED [stage-1 5/9] RUN chown -R 1250:1250 /home/ && chmod -R 0700 /home/                                                                                   0.0s
 => CACHED [stage-1 6/9] RUN pip install -r /requirements.txt                                                                                                    0.0s
 => [stage-1 7/9] ADD . /home/appuser/wwwroot                                                                                                                    0.1s
 => [stage-1 8/9] WORKDIR /home/appuser/wwwroot/HttpUploadTrigger/                                                                                               0.0s
 => ERROR [stage-1 9/9] RUN pip install pytest && pytest --verbose                                                                                               1.0s
------
 > [stage-1 9/9] RUN pip install pytest && pytest --verbose:
#15 0.785 Defaulting to user installation because normal site-packages is not writeable
#15 0.815 Requirement already satisfied: pytest in /usr/local/lib/python3.8/site-packages (6.2.2)
#15 0.825 Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in /usr/local/lib/python3.8/site-packages (from pytest) (0.13.1)
#15 0.827 Requirement already satisfied: packaging in /usr/local/lib/python3.8/site-packages (from pytest) (20.9)
#15 0.828 Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.8/site-packages (from pytest) (20.3.0)
#15 0.829 Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.8/site-packages (from pytest) (1.10.0)
#15 0.830 Requirement already satisfied: toml in /usr/local/lib/python3.8/site-packages (from pytest) (0.10.2)
#15 0.830 Requirement already satisfied: iniconfig in /usr/local/lib/python3.8/site-packages (from pytest) (1.1.1)
#15 0.852 Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.8/site-packages (from packaging->pytest) (2.4.7)
#15 1.007 /bin/sh: 1: pytest: not found
------
executor failed running [/bin/sh -c pip install pytest && pytest --verbose]: exit code: 127

It looks like the pytest package cannot be installed because some permissions denied, perhaps because I am running the container using appuser

I am not sure about this

---

UPDATE

I redefined the Dockerfile by creating a virtual environment at the first stage and copying it to the final stage such as @Itamar Turner-Trauring point me out in his answer below.

The Dockefile is:

FROM mcr.microsoft.com/azure-functions/python:3.0-python3.8 as intermediate

RUN apt-get update && \
    apt-get install -y apt-utils && apt-get install -y git && \
    apt-get install -y --no-install-recommends build-essential gcc && \
    wget https://packages.microsoft.com/config/ubuntu/20.10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg -i packages-microsoft-prod.deb && \
    apt-get update; \
    apt-get install -y apt-transport-https && \
    apt-get update && \
    apt-get install -y dotnet-sdk-5.0

ARG SPECKLE_ENFORCE_SSL
ENV SPECKLE_ENFORCE_SSL=false
ARG ARTIFACTS_KEYRING_NONINTERACTIVE_MODE
ENV ARTIFACTS_KEYRING_NONINTERACTIVE_MODE=true

# AZ_DEVOPS_TOKEN is a PAT from Azure DevOps with Packaging and Build scope privileges
ARG AZ_DEVOPS_TOKEN
ENV AZ_DEVOPS_TOKEN=$AZ_DEVOPS_TOKEN
ENV PYTHONUNBUFFERED 1

ENV VIRTUAL_ENV=/opt/env
RUN python -m venv $VIRTUAL_ENV
# Make sure we use the virtualenv:
ENV PATH="$VIRTUAL_ENV/bin::/home/.local/bin:$PATH"

RUN pip install --upgrade pip && \
    pip install pyyaml numpy lxml artifacts-keyring && \
    pip install -i https://$AZ_DEVOPS_TOKEN@pkgs.dev.azure.com/corporateroot/DataFusr%20FEM-client/_packaging/rhdhv_fem/pypi/simple/ --no-cache-dir rhdhv-fem

# Ditch the intermediate layer
FROM mcr.microsoft.com/azure-functions/python:3.0-python3.8
RUN useradd --create-home appuser
COPY --from=intermediate /home/.local/ /home/.local/
RUN chown -R appuser /home/.local/
USER appuser

ENV AzureWebJobsScriptRoot=/home/appuser/wwwroot \
  AzureFunctionsJobHost__Logging__Console__IsEnabled=true

# COPY --from=intermediate /usr/local/lib/python3.8/site-packages/ /usr/local/lib/python3.8/site-packages/
COPY --from=intermediate --chown=appuser $VIRTUAL_ENV /opt/venv
ADD requirements.txt /
# Make sure we use the virtualenv:
ENV PATH="/opt/venv/bin:/home/.local/bin:$PATH"
RUN pip install -r /requirements.txt

ADD . /home/appuser/wwwroot
WORKDIR /home/appuser/wwwroot/HttpUploadTrigger/
RUN pip install pytest && pytest --verbose

When I run the build I got this output when executing the last line RUN pip install pytest && pytest --verbose:

 [stage-1 10/10] RUN pip install pytest && pytest --verbose:
#17 1.053 WARNING: The directory '/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
#17 1.055 Defaulting to user installation because normal site-packages is not writeable
#17 1.394 Collecting pytest
#17 1.589   Downloading pytest-6.2.2-py3-none-any.whl (280 kB)
#17 2.024 Collecting attrs>=19.2.0
#17 2.050   Downloading attrs-20.3.0-py2.py3-none-any.whl (49 kB)

Despite that the workflow keeps going, but finally I got this error

17 2.811 Successfully installed attrs-20.3.0 iniconfig-1.1.1 packaging-20.9 pluggy-0.13.1 py-1.10.0 pyparsing-2.4.7 pytest-6.2.2 toml-0.10.2
#17 3.037 WARNING: You are using pip version 20.3.3; however, version 21.0.1 is available.
#17 3.037 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
#17 3.434 ============================= test session starts ==============================
#17 3.434 platform linux -- Python 3.8.7, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/local/bin/python
#17 3.434 cachedir: .pytest_cache
#17 3.434 rootdir: /home/appuser/wwwroot/HttpUploadTrigger
#17 3.434 collecting ... collected 0 items / 1 error
#17 3.778
#17 3.778 ==================================== ERRORS ====================================
#17 3.778 _____________________ ERROR collecting test_httptrigger.py _____________________
#17 3.778 ImportError while importing test module '/home/appuser/wwwroot/HttpUploadTrigger/test_httptrigger.py'.
#17 3.778 Hint: make sure your test modules/packages have valid Python names.
#17 3.778 Traceback:
#17 3.778 /usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
#17 3.778     return _bootstrap._gcd_import(name[level:], package, level)
#17 3.778 __init__.py:16: in <module>
#17 3.778     from rhdhv_fem.fem_SCIAutils import _fem_scia_to_fem
#17 3.778 E   ModuleNotFoundError: No module named 'rhdhv_fem'

It is indicating that my private package (that one I install from https://$AZ_DEVOPS_TOKEN@pkgs.dev.azure.com/corporateroot/DataFusr%20FEM-client/_packaging/rhdhv_fem/pypi/simple/ --no-cache-dir rhdhv-fem) couldn't be readed from the pytest execution despite it was installed

Notice I've also needed to copy the `/home/.local/ directory from my first intermediate stage, at the beginning of the final stage and give to my appuser permissions such as:

# Ditch the intermediate layer
FROM mcr.microsoft.com/azure-functions/python:3.0-python3.8
RUN useradd --create-home appuser
COPY --from=intermediate /home/.local/ /home/.local/
RUN chown -R appuser /home/.local/
USER appuser

I am not sure since I am copying the virtual environment and I am not using pip’s --user option, why the files are being installed at .local directory such as is highlighted here ...

Answer 1

If you want to install in one image and run in the other, it's often easier to install into a virtualenv and copy the virtualenv, because then you get all the files, e.g. data files and executables and such like. Copying site-packages doesn't get you that.

You can then chown the virtualenv to your new user as you copy it.

Something like

FROM python:3.9-slim AS compile-image
RUN apt-get update
RUN apt-get install -y --no-install-recommends build-essential gcc

RUN python -m venv /opt/venv
# Make sure we use the virtualenv:
ENV PATH="/opt/venv/bin:$PATH"

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY setup.py .
COPY myapp/ .
RUN pip install .

FROM python:3.9-slim AS build-image
RUN useradd --create-home appuser
USER appuser
COPY --from=compile-image --chown=appuser /opt/venv /opt/venv

# Make sure we use the virtualenv:
ENV PATH="/opt/venv/bin:$PATH"
CMD ['myapp']

Note the use of --chown in COPY to ensure it changes ownership as you copy it. (Based on https://pythonspeed.com/articles/activate-virtualenv-dockerfile/, https://pythonspeed.com/articles/multi-stage-docker-python/ https://pythonspeed.com/articles/root-capabilities-docker-security/)

Answer 2

You can add USER some_user to your Dockerfile o run with a specific user

cat Dockerfile
...    
USER site
RUN echo 'RUNNING as $USER'

I ran the image

Or at execution

docker run -it --user site image_name bash

I don't know if there are limitations in the USER instruction só you could try add RUN root in the intermediate container and then RUN site in the final container.

, maybe the user you want is www-data