Reputation: 3407

Integrating Python Poetry with Docker

Can you give me an example of a Dockerfile in which I can install all the packages I need from poetry.lock and pyproject.toml into my image/container from Docker?

Upvotes: 339

Answers (16)

sobolevn

Reputation: 18070

There are several things to keep in mind when using Poetry together with Docker.

Installation

Official way to install Poetry is via:

curl -sSL https://install.python-poetry.org | python3 -

This way allows Poetry and its dependencies to be isolated from your dependencies.

You can also use pip install 'poetry==$POETRY_VERSION'. But, this will install Poetry and its dependencies into your main site-packages/. It might not be ideal.

Also, pin this version in your pyproject.toml as well:

[build-system]
# Should be the same as `$POETRY_VERSION`:
requires = ["poetry-core>=1.6"]
build-backend = "poetry.core.masonry.api"

It will protect you from version mismatch between your local and Docker environments.

Caching dependencies

We want to cache our requirements and only reinstall them when pyproject.toml or poetry.lock files change. Otherwise builds will be slow. To achieve working cache layer we should put:

COPY poetry.lock pyproject.toml /code/

after Poetry is installed, but before any other files are added.

Virtualenv

The next thing to keep in mind is virtualenv creation. We do not need it in Docker. It is already isolated. So, we use POETRY_VIRTUALENVS_CREATE=false or poetry config virtualenvs.create false setting to turn it off.

Development vs. Production

If you use the same Dockerfile for both development and production as I do, you will need to install different sets of dependencies based on some environment variable:

poetry install $(test "$YOUR_ENV" == production && echo "--only=main")

This way $YOUR_ENV will control which dependencies set will be installed: all (default) or production only with --only=main flag.

You may also want to add some more options for better experience:

--no-interaction not to ask any interactive questions
--no-ansi flag to make your output more log friendly

Result

You will end up with something similar to:

FROM python:3.11.5-slim-bookworm

ARG YOUR_ENV

ENV YOUR_ENV=${YOUR_ENV} \
  PYTHONFAULTHANDLER=1 \
  PYTHONUNBUFFERED=1 \
  PYTHONHASHSEED=random \
  PIP_NO_CACHE_DIR=off \
  PIP_DISABLE_PIP_VERSION_CHECK=on \
  PIP_DEFAULT_TIMEOUT=100 \
  # Poetry's configuration:
  POETRY_NO_INTERACTION=1 \
  POETRY_VIRTUALENVS_CREATE=false \
  POETRY_CACHE_DIR='/var/cache/pypoetry' \
  POETRY_HOME='/usr/local' \
  POETRY_VERSION=1.7.1
  # ^^^
  # Make sure to update it!

# System deps:
RUN curl -sSL https://install.python-poetry.org | python3 -

# Copy only requirements to cache them in docker layer
WORKDIR /code
COPY poetry.lock pyproject.toml /code/

# Project initialization:
RUN poetry install $(test "$YOUR_ENV" == production && echo "--only=main") --no-interaction --no-ansi

# Creating folders, and files for a project:
COPY . /code

You can find a fully working real-life example here.

Upvotes: 484

rjurney

Reputation: 5140

Here is a recipe using the official installer bash script and the continuumio/miniconda Docker image. This image is a good place to start with your Pythonic Docker stack.

FROM continuumio/miniconda3

RUN apt update && \
    apt-get install -y curl && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

...

# Install poetry to /root/.local/bin
ENV POETRY_VIRTUALENVS_CREATE=false \
    POETRY_VERSION=1.8.3

# Install Python packages via Poetry
WORKDIR /app
RUN curl -sSL https://install.python-poetry.org | python3 -
ENV PATH="/root/.local/bin:$PATH"
COPY pyproject.toml poetry.lock /app/

# Skips dev packagesL --no-dev, doesn't install a package: --no-root
RUN poetry config virtualenvs.create false && \
    poetry config installer.max-workers 10 && \
    poetry install --no-dev --no-interaction --no-ansi --no-root -vvv && \
    poetry cache clear pypi --all -n

However, in some corporate environments, you can't run curl to the hostname install.python-poetry.org because you use something like Artifactory to install packages from a local repository. Here is a recipe for installing pipx to install poetry.

FROM continuumio/miniconda3:latest

RUN apt update && \
    apt upgrade -y && \
    apt install curl -y && \
    apt clean && \
    rm -rf /var/lib/apt/lists/*

# Install pipx so we can install poetry inside the firewall
RUN python3 -m pip install --index-url "https://pypi-repository-url/folder" --user pipx
ENV PATH=/root/.local/bin:$PATH

# Install poetry to /root/.local/bin
ENV POETRY_VIRTUALENVS_CREATE=false \
    POETRY_VERSION=1.8.3
RUN pipx install --index-url "https://pypi-repository-url/folder" poetry

# Copy the poetry.lock and pyproject.toml files
WORKDIR /app
COPY pyproject.toml poetry.lock ./app/

# Skips dev packagesL --no-dev, doesn't install a package: --no-root
RUN poetry config virtualenvs.create false && \
    poetry config installer.max-workers 10 && \
    poetry install --no-dev --no-interaction --no-ansi --no-root -vvv && \
    poetry cache clear pypi --all -n

Upvotes: 1

Vincent Claes

Reputation: 4768

I added this to my Dockerfile and it worked

RUN pip3 install pipx
RUN pipx install poetry
ENV PATH="/root/.local/bin:${PATH}"

Upvotes: 0

Claudio

Reputation: 3928

Multi-stage Docker build with Poetry and venv

Update (2024-03-16)

This has become much easier over the past years. These days I'd use Poetry's bundle plugin to install the application into a virtual environment, then copy the virtual environment into a distroless image. Install Poetry with pipx, which is packaged by Debian. (You likely want to pin Poetry to avoid breakage when your project isn't compatible with a new Poetry release.) Use the option --only=main when bundling to omit development dependencies.

FROM debian:12-slim AS builder
RUN apt-get update && \
    apt-get install --no-install-suggests --no-install-recommends --yes pipx
ENV PATH="/root/.local/bin:${PATH}"
RUN pipx install poetry
RUN pipx inject poetry poetry-plugin-bundle
WORKDIR /src
COPY . .
RUN poetry bundle venv --python=/usr/bin/python3 --only=main /venv

FROM gcr.io/distroless/python3-debian12
COPY --from=builder /venv /venv
ENTRYPOINT ["/venv/bin/my-awesome-app"]

Original Answer

Do not disable virtualenv creation. Virtualenvs serve a purpose in Docker builds, because they provide an elegant way to leverage multi-stage builds. In a nutshell, your build stage installs everything into the virtualenv, and the final stage just copies the virtualenv over into a small image.

Use poetry export and install your pinned requirements first, before copying your code. This will allow you to use the Docker build cache, and never reinstall dependencies just because you changed a line in your code.

Do not use poetry install to install your code, because it will perform an editable install. Instead, use poetry build to build a wheel, and then pip-install that into your virtualenv. (Thanks to PEP 517, this whole process could also be performed with a simple pip install ., but due to build isolation you would end up installing another copy of Poetry.)

Here's an example Dockerfile installing a Flask app into an Alpine image, with a dependency on Postgres. This example uses an entrypoint script to activate the virtualenv. But generally, you should be fine without an entrypoint script because you can simply reference the Python binary at /venv/bin/python in your CMD instruction.

Dockerfile

FROM python:3.7.6-alpine3.11 as base

ENV PYTHONFAULTHANDLER=1 \
    PYTHONHASHSEED=random \
    PYTHONUNBUFFERED=1

WORKDIR /app

FROM base as builder

ENV PIP_DEFAULT_TIMEOUT=100 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PIP_NO_CACHE_DIR=1 \
    POETRY_VERSION=1.0.5

RUN apk add --no-cache gcc libffi-dev musl-dev postgresql-dev
RUN pip install "poetry==$POETRY_VERSION"
RUN python -m venv /venv

COPY pyproject.toml poetry.lock ./
RUN poetry export -f requirements.txt | /venv/bin/pip install -r /dev/stdin

COPY . .
RUN poetry build && /venv/bin/pip install dist/*.whl

FROM base as final

RUN apk add --no-cache libffi libpq
COPY --from=builder /venv /venv
COPY docker-entrypoint.sh wsgi.py ./
CMD ["./docker-entrypoint.sh"]

docker-entrypoint.sh

#!/bin/sh

set -e

. /venv/bin/activate

while ! flask db upgrade
do
     echo "Retry..."
     sleep 1
done

exec gunicorn --bind 0.0.0.0:5000 --forwarded-allow-ips='*' wsgi:app

wsgi.py

import your_app

app = your_app.create_app()

Upvotes: 206

bneijt

Reputation: 175

I've created a solution using a lock package (package which depends on all versions in the lock file). This results in a clean pip-only install without requirements files.

Steps are: build the package, build the lock package, copy both wheels into your container, install both wheels with pip.

Installation is: poetry add --dev poetry-lock-package

Steps outside of docker build are:

poetry build
poetry run poetry-lock-package --build

Then your Dockerfile should contain:

FROM python:3-slim

COPY dist/*.whl /

RUN pip install --no-cache-dir /*.whl \
    && rm -rf /*.whl

CMD ["python", "-m", "entry_module"]

To allow this to work for multiple platforms, the first steps can be done in a first stage of a multistage build. Example:

FROM python:alpine AS builder

WORKDIR /app

RUN pip install --no-cache-dir --upgrade pip \
 && pip install --no-cache-dir poetry

COPY . ./

RUN poetry add --group dev poetry-lock-package
RUN poetry build
RUN poetry run poetry-lock-package --build

FROM python:alpine

ENV PYTHONUNBUFFERED 1

WORKDIR /app

COPY --from=builder /app/dist/*.whl /

RUN pip install --no-cache-dir /*.whl \
    && rm -rf /*.whl

CMD [ "python", "-m", "entry_module" ]

Upvotes: 10

GoForth

Reputation: 656

The other answers were good but I had to make some modifications based on the following requirements I had:

I wanted a small image and so wanted to use Alpine.
I wanted to be sure not to run the final image as root.
I was particularly focused on being able to run a Poetry script from my pyproject.toml by name.

For example, if I have this script in the pyproject.toml:

...
[tool.poetry.scripts]
my_tool = "my_tool.cli.cli:start"
...

Then, I wanted my_tool (a CLI) to be my Dockerfile ENTRYPOINT so that the arguments provided by the container commands would be arguments to my CLI. This solution accomplished exactly what I was looking for:

# Stage - base
FROM python:3.11-alpine3.18 as base

ENV PYTHONFAULTHANDLER=1 \
    PYTHONHASHSEED=random \
    PYTHONUNBUFFERED=1

WORKDIR /app

# Stage - builder
FROM python:3.11-alpine3.18 as builder

ENV PIP_DEFAULT_TIMEOUT=100 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    POETRY_VERSION=1.7.0

RUN pip install poetry==$POETRY_VERSION

WORKDIR /app

RUN python -m venv /venv

COPY pyproject.toml poetry.lock ./
RUN . /venv/bin/activate && poetry install --no-dev --no-root

COPY . .
RUN . /venv/bin/activate && poetry build

# Stage - release
FROM base as release

# install sudo as root
RUN apk add --update sudo

# add new user
ENV USER=appuser
RUN adduser -D $USER \
        && echo "$USER ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/$USER \
        && chmod 0440 /etc/sudoers.d/$USER

ENV PATH="/venv/bin:$PATH"

COPY --from=builder /venv /venv
COPY --from=builder /app/dist .
RUN chown -hR $USER /venv

RUN . /venv/bin/activate && pip install *.whl

USER $USER

ENTRYPOINT ["my_cli"]

Upvotes: 0

Max Pfeiffer

Reputation: 99

I provide a Poetry docker image to the community. This image is always available for the latest three Poetry versions and different Python versions. You can pick your favorite:

https://hub.docker.com/r/pfeiffermax/python-poetry

You can check the Docker file for the practices I applied there. It's quite simple: https://github.com/max-pfeiffer/python-poetry/blob/main/build/Dockerfile

# References: using official Python images
# https://hub.docker.com/_/python
ARG OFFICIAL_PYTHON_IMAGE
FROM ${OFFICIAL_PYTHON_IMAGE}
ARG POETRY_VERSION

LABEL maintainer="Max Pfeiffer <[email protected]>"

# References:
# https://pip.pypa.io/en/stable/topics/caching/#avoiding-caching
# https://pip.pypa.io/en/stable/cli/pip/?highlight=PIP_NO_CACHE_DIR#cmdoption-no-cache-dir
# https://pip.pypa.io/en/stable/cli/pip/?highlight=PIP_DISABLE_PIP_VERSION_CHECK#cmdoption-disable-pip-version-check
# https://pip.pypa.io/en/stable/cli/pip/?highlight=PIP_DEFAULT_TIMEOUT#cmdoption-timeout
# https://pip.pypa.io/en/stable/topics/configuration/#environment-variables
# https://python-poetry.org/docs/#installation

ENV PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100 \
    POETRY_VERSION=${POETRY_VERSION} \
    POETRY_HOME="/opt/poetry"

ENV PATH="$POETRY_HOME/bin:$PATH"

# https://python-poetry.org/docs/#osx--linux--bashonwindows-install-instructions
RUN apt-get update \
    && apt-get install --no-install-recommends -y \
        build-essential \
        curl \
    && curl -sSL https://install.python-poetry.org | python - \
    && apt-get purge --auto-remove -y \
      build-essential \
      curl

This image I use as base image in two other projects where you can see how to utilise Poetry for creating virtual environments and run Python applications using Uvicorn and/or Gunicorn application servers :

Dockerfile of first image: https://github.com/max-pfeiffer/uvicorn-poetry/blob/main/build/Dockerfile

# The Poetry installation is provided through the base image. Please check the
# base image if you interested in the details.
# Base image: https://hub.docker.com/r/pfeiffermax/python-poetry
# Dockerfile: https://github.com/max-pfeiffer/python-poetry/blob/main/build/Dockerfile
ARG BASE_IMAGE
FROM ${BASE_IMAGE}
ARG APPLICATION_SERVER_PORT

LABEL maintainer="Max Pfeiffer <[email protected]>"

    # https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUNBUFFERED
ENV PYTHONUNBUFFERED=1 \
    # https://docs.python.org/3/using/cmdline.html#envvar-PYTHONDONTWRITEBYTECODE
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONPATH=/application_root \
    # https://python-poetry.org/docs/configuration/#virtualenvsin-project
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    POETRY_CACHE_DIR="/application_root/.cache" \
    VIRTUAL_ENVIRONMENT_PATH="/application_root/.venv" \
    APPLICATION_SERVER_PORT=$APPLICATION_SERVER_PORT

# Adding the virtual environment to PATH in order to "activate" it.
# https://docs.python.org/3/library/venv.html#how-venvs-work
ENV PATH="$VIRTUAL_ENVIRONMENT_PATH/bin:$PATH"

# Principle of least privilege: create a new user for running the application
RUN groupadd -g 1001 python_application && \
    useradd -r -u 1001 -g python_application python_application

# Set the WORKDIR to the application root.
# https://www.uvicorn.org/settings/#development
# https://docs.docker.com/engine/reference/builder/#workdir
WORKDIR ${PYTHONPATH}
RUN chown python_application:python_application ${PYTHONPATH}

# Create cache directory and set permissions because user 1001 has no home
# and poetry cache directory.
# https://python-poetry.org/docs/configuration/#cache-directory
RUN mkdir ${POETRY_CACHE_DIR} && chown python_application:python_application ${POETRY_CACHE_DIR}

# Document the exposed port
# https://docs.docker.com/engine/reference/builder/#expose
EXPOSE ${APPLICATION_SERVER_PORT}

# Use the unpriveledged user to run the application
USER 1001

# Run the uvicorn application server.
CMD exec uvicorn --workers 1 --host 0.0.0.0 --port $APPLICATION_SERVER_PORT app.main:app

If you structured it like this the Dockerfile of a sample application can be as simple as this doing a multistage build: https://github.com/max-pfeiffer/uvicorn-poetry/blob/main/examples/fast_api_multistage_build/Dockerfile

# Be aware that you need to specify these arguments before the first FROM
# see: https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
ARG BASE_IMAGE=pfeiffermax/uvicorn-poetry:3.0.0-python3.10.9-slim-bullseye@sha256:cdd772b5e6e3f2feb8d38f3ca7af9b955c886a86a4aecec99bc43897edd8bcbe
FROM ${BASE_IMAGE} as dependencies-build-stage

# install [tool.poetry.dependencies]
# this will install virtual environment into /.venv because of POETRY_VIRTUALENVS_IN_PROJECT=true
# see: https://python-poetry.org/docs/configuration/#virtualenvsin-project
COPY ./poetry.lock ./pyproject.toml /application_root/
RUN poetry install --no-interaction --no-root --without dev

FROM ${BASE_IMAGE} as production-image

# Copy virtual environment
COPY --chown=python_application:python_application --from=dependencies-build-stage /application_root/.venv /application_root/.venv

# Copy application files
COPY --chown=python_application:python_application /app /application_root/app/

Upvotes: 7

Jeffrey04

Reputation: 6338

This is a minor revision to the answer provided by @Claudio, which uses the new poetry install --no-root feature as described by @sobolevn in his answer.

In order to force poetry to install dependencies into a specific virtualenv, one needs to first enable it.

. /path/to/virtualenv/bin/activate && poetry install

Therefore adding these into @Claudio's answer we have

FROM python:3.10-slim as base

ENV PYTHONFAULTHANDLER=1 \
    PYTHONHASHSEED=random \
    PYTHONUNBUFFERED=1

WORKDIR /app

FROM base as builder

ENV PIP_DEFAULT_TIMEOUT=100 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PIP_NO_CACHE_DIR=1 \
    POETRY_VERSION=1.3.1

RUN pip install "poetry==$POETRY_VERSION"

COPY pyproject.toml poetry.lock README.md ./
# if your project is stored in src, uncomment line below
# COPY src ./src
# or this if your file is stored in $PROJECT_NAME, assuming `myproject`
# COPY myproject ./myproject
RUN poetry config virtualenvs.in-project true && \
    poetry install --only=main --no-root && \
    poetry build

FROM base as final

COPY --from=builder /app/.venv ./.venv
COPY --from=builder /app/dist .
COPY docker-entrypoint.sh .

RUN ./.venv/bin/pip install *.whl
CMD ["./docker-entrypoint.sh"]

If you need to use this for development purpose, you add or remove the --no-dev by replacing this line

RUN . /venv/bin/activate && poetry install --no-dev --no-root

to something like this as shown in @sobolevn's answer

RUN . /venv/bin/activate && poetry install --no-root $(test "$YOUR_ENV" == production && echo "--no-dev")

after adding the appropriate environment variable declaration.

The example uses debian-slim's as base, however, adapting this to alpine-based image should be a trivial task.

Upvotes: 47

Nikhil Akki

Reputation: 11

Dockerfile for my python apps looks like this -

FROM python:3.10-alpine
RUN apk update && apk upgrade
RUN pip install -U pip poetry==1.1.13
WORKDIR /app
COPY . .
RUN poetry export --without-hashes --format=requirements.txt > requirements.txt
RUN pip install -r requirements.txt
EXPOSE 8000
ENTRYPOINT [ "python" ]
CMD ["main.py"]

Upvotes: 0

Krizza

Reputation: 641

My Dockerfile based on @lmiguelvargasf's answer. Do refer to his post for a more detailed explanation. The only significant changes I have are the following:

I am now using the latest official installer install-poetry.py instead of the deprecated get-poetry.py as recommended in their official documentation. I'm also installing a specific version using the --version flag but you can alternatively use the environment variable POETRY_VERSION. More info on their official docs!
The PATH I use is /root/.local/bin:$PATH instead of ${PATH}:/root/.poetry/bin from OP's Dockerfile

FROM python:3.10.4-slim-buster

ENV PYTHONDONTWRITEBYTECODE 1 \
    PYTHONUNBUFFERED 1

RUN apt-get update \
    && apt-get install curl -y \
    && curl -sSL https://install.python-poetry.org | python - --version 1.1.13

ENV PATH="/root/.local/bin:$PATH"

WORKDIR /usr/app

COPY pyproject.toml poetry.lock ./

RUN poetry config virtualenvs.create false \
    && poetry install --no-dev --no-interaction --no-ansi

COPY ./src ./

EXPOSE 5000

CMD [ "poetry", "run", "gunicorn", "-b", "0.0.0.0:5000", "test_poetry.app:create_app()" ]

Upvotes: 13

guyskk

Reputation: 2736

Use docker multiple stage build and python slim image, export poetry lock to requirements.txt, then install via pip inside virtualenv.

It has smallest size, not require poetry in runtime image, pin the versions of everything.

FROM python:3.9.7 as base
ENV PIP_DISABLE_PIP_VERSION_CHECK=1
WORKDIR /app

FROM base as poetry
RUN pip install poetry==1.1.12
COPY poetry.lock pyproject.toml /app/
RUN poetry export -o requirements.txt

FROM base as build
COPY --from=poetry /app/requirements.txt /tmp/requirements.txt
RUN python -m venv .venv && \
    .venv/bin/pip install 'wheel==0.36.2' && \
    .venv/bin/pip install -r /tmp/requirements.txt

FROM python:3.9.7-slim as runtime
ENV PIP_DISABLE_PIP_VERSION_CHECK=1
WORKDIR /app
ENV PATH=/app/.venv/bin:$PATH
COPY --from=build /app/.venv /app/.venv
COPY . /app

Upvotes: 17

gbw

Reputation: 51

Here's a different approach that leaves Poetry intact so you can still use poetry add etc. This is good if you're using a VS Code devcontainer.

In short, install Poetry, let Poetry create the virtual environment, then enter the virtual environment every time you start a new shell by modifying .bashrc.

FROM ubuntu:20.04

RUN apt-get update && apt-get install -y python3 python3-pip curl

# Use Python 3 for `python`, `pip`
RUN    update-alternatives --install /usr/bin/python  python  /usr/bin/python3 1 \
    && update-alternatives --install /usr/bin/pip     pip     /usr/bin/pip3    1

# Install Poetry
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3 -
ENV PATH "$PATH:/root/.local/bin/"

# Install Poetry packages (maybe remove the poetry.lock line if you don't want/have a lock file)
COPY pyproject.toml ./
COPY poetry.lock ./
RUN poetry install --no-interaction

# Provide a known path for the virtual environment by creating a symlink
RUN ln -s $(poetry env info --path) /var/my-venv

# Clean up project files. You can add them with a Docker mount later.
RUN rm pyproject.toml poetry.lock

# Hide virtual env prompt
ENV VIRTUAL_ENV_DISABLE_PROMPT 1

# Start virtual env when bash starts
RUN echo 'source /var/my-venv/bin/activate' >> ~/.bashrc

Reminder that there's no need to avoid the virtualenv. It doesn't affect performance and Poetry isn't really designed to work without them.

EDIT: @Davos points out that this doesn't work unless you already have a pyproject.toml and poetry.lock file. If you need to handle that case, you might be able to use this workaround which should work whether or not those files exist.

COPY pyproject.toml* ./
COPY poetry.lock* ./
RUN poetry init --no-interaction; (exit 0) # Does nothing if pyproject.toml exists
RUN poetry install --no-interaction

Upvotes: 5

lmiguelvargasf

Reputation: 69655

TL;DR

I have been able to set up poetry for a Django project using postgres. After doing some research, I ended up with the following Dockerfile:

FROM python:slim

# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE 1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED 1

# Install and setup poetry
RUN pip install -U pip \
    && apt-get update \
    && apt install -y curl netcat \
    && curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
ENV PATH="${PATH}:/root/.poetry/bin"

WORKDIR /usr/src/app
COPY . .
RUN poetry config virtualenvs.create false \
  && poetry install --no-interaction --no-ansi

# run entrypoint.sh
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]

This is the content of entrypoint.sh:

#!/bin/sh

if [ "$DATABASE" = "postgres" ]
then
    echo "Waiting for postgres..."

    while ! nc -z $SQL_HOST $SQL_PORT; do
      sleep 0.1
    done

    echo "PostgreSQL started"
fi

python manage.py migrate

exec "$@"

Detailed Explanation

Some points to notice:

I have decide to use slim instead of alpine as tag for the python image because even though alpine images are supposed to reduce the size of Docker images and speed up the build, with Python, you can actually end up with a bit larger image and that takes a while to build (read this article for more info).
Using this configuration builds containers faster than using the alpine image because I do not need to add some extra packages to install Python packages properly.
I am installing poetry directly from the URL provided in the documentation. I am aware of the warnings provided by sobolevn. However, I consider that it is better in the long term to use the lates version of poetry by default than relying on an environment variable that I should update periodically.
Updating the environment variable PATH is crucial. Otherwise, you will get an error saying that poetry was not found.
Dependencies are installed directly in the python interpreter of the container. It does not create poetry to create a virtual environment before installing the dependencies.

In case you need the alpine version of this Dockerfile:

FROM python:alpine

# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE 1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED 1

# Install dev dependencies
RUN apk update \
    && apk add curl postgresql-dev gcc python3-dev musl-dev openssl-dev libffi-dev

# Install poetry
RUN pip install -U pip \
    && curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
ENV PATH="${PATH}:/root/.poetry/bin"

WORKDIR /usr/src/app
COPY . .
RUN poetry config virtualenvs.create false \
  && poetry install --no-interaction --no-ansi

# run entrypoint.sh
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]

Notice that the alpine version needs some dependencies postgresql-dev gcc python3-dev musl-dev openssl-dev libffi-dev to work properly.

Upvotes: 32

Mattia Fantoni

Reputation: 955

I see all the answers here are using the pip way to install Poetry to avoid version issue. The official way to install poetry read POETRY_VERSION env variable if defined to install the most appropriate version.

There is an issue in github here and I think the solution from this ticket is quite interesting:

# `python-base` sets up all our shared environment variables
FROM python:3.8.1-slim as python-base

    # python
ENV PYTHONUNBUFFERED=1 \
    # prevents python creating .pyc files
    PYTHONDONTWRITEBYTECODE=1 \
    \
    # pip
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100 \
    \
    # poetry
    # https://python-poetry.org/docs/configuration/#using-environment-variables
    POETRY_VERSION=1.0.3 \
    # make poetry install to this location
    POETRY_HOME="/opt/poetry" \
    # make poetry create the virtual environment in the project's root
    # it gets named `.venv`
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    # do not ask any interactive question
    POETRY_NO_INTERACTION=1 \
    \
    # paths
    # this is where our requirements + virtual environment will live
    PYSETUP_PATH="/opt/pysetup" \
    VENV_PATH="/opt/pysetup/.venv"


# prepend poetry and venv to path
ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"


# `builder-base` stage is used to build deps + create our virtual environment
FROM python-base as builder-base
RUN apt-get update \
    && apt-get install --no-install-recommends -y \
        # deps for installing poetry
        curl \
        # deps for building python deps
        build-essential

# install poetry - respects $POETRY_VERSION & $POETRY_HOME
RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

# copy project requirement files here to ensure they will be cached.
WORKDIR $PYSETUP_PATH
COPY poetry.lock pyproject.toml ./

# install runtime deps - uses $POETRY_VIRTUALENVS_IN_PROJECT internally
RUN poetry install --no-dev


# `development` image is used during development / testing
FROM python-base as development
ENV FASTAPI_ENV=development
WORKDIR $PYSETUP_PATH

# copy in our built poetry + venv
COPY --from=builder-base $POETRY_HOME $POETRY_HOME
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH

# quicker install as runtime deps are already installed
RUN poetry install

# will become mountpoint of our code
WORKDIR /app

EXPOSE 8000
CMD ["uvicorn", "--reload", "main:app"]


# `production` image used for runtime
FROM python-base as production
ENV FASTAPI_ENV=production
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH
COPY ./app /app/
WORKDIR /app
CMD ["gunicorn", "-k", "uvicorn.workers.UvicornWorker", "main:app"]

Upvotes: 6

maciek

Reputation: 3354

That's minimal configuration that works for me:

FROM python:3.7

ENV PIP_DISABLE_PIP_VERSION_CHECK=on

RUN pip install poetry

WORKDIR /app
COPY poetry.lock pyproject.toml /app/

RUN poetry config virtualenvs.create false
RUN poetry install --no-interaction

COPY . /app

Note that it is not as safe as @sobolevn's configuration.

As a trivia I'll add that if editable installs will be possible for pyproject.toml projects, a line or two could be deleted:

FROM python:3.7

ENV PIP_DISABLE_PIP_VERSION_CHECK=on

WORKDIR /app
COPY poetry.lock pyproject.toml /app/

RUN pip install -e .

COPY . /app

Upvotes: 23

funky-future

Reputation: 3958

Here's a stripped example where first a layer with the dependencies (that is only build when these changed) and then one with the full source code is added to an image. Setting poetry to install into the global site-packages leaves a configuration artifact that could also be removed.

FROM python:alpine

WORKDIR /app

COPY poetry.lock pyproject.toml ./
RUN pip install --no-cache-dir --upgrade pip \
 && pip install --no-cache-dir poetry \
 \
 && poetry config settings.virtualenvs.create false \
 && poetry install --no-dev \
 \
 && pip uninstall --yes poetry \

COPY . ./

Upvotes: 16

Integrating Python Poetry with Docker

Answers (16)

Installation

Caching dependencies

Virtualenv

Development vs. Production

Result

TL;DR

Detailed Explanation

Related Questions