CODEWITHSUNDEEP
pythondocker
Adam
Adam

Reputation: 4172

Docker volume bind breaks python script CLI entry_point

Alright so I have a fairly basic python click CLI application that when ran with a bound volume in a container (live dev on files) appears to break the entry_point I have in setup.py

Running either of the following commands

$ docker run -it -v $(pwd):/opt/app gdax
$ docker run -it --mount src=$(pwd),target=/opt/app,type=bind gdax

I get the following

Traceback (most recent call last):
  File "/usr/local/bin/gdax", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3144, in <module>
    @_call_aside
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3128, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3157, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 666, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 984, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 870, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'gdax-cli' distribution was not found and is required by the application

Dockerfile

FROM python:3.6

RUN mkdir /opt/app
COPY . /opt/app


WORKDIR /opt/app

RUN pip install --editable .

CMD ["gdax", "--help"]

setup.py

from setuptools import setup


setup(
    name='gdax-cli',
    version='0.1',
    py_modules=['app'],
    install_requires=[
        'click==6.7',
        'gdax==1.0.6'
    ],
    entry_points='''
        [console_scripts]
        gdax=app:cli
    ''',
)

/usr/local/bin/gdax <- on container

#!/usr/local/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'gdax-cli','console_scripts','gdax'
__requires__ = 'gdax-cli'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('gdax-cli', 'console_scripts', 'gdax')()
    )

If I don't bind the volume I can execute the script just fine but I lose the ability to edit the files on the host machine. I'm fairly confident this has something to do with how binding "overwrites" the files and will break /usr/local/bin/gdax ability to load the files which is placed there by setuptools. Is there anyway to circumvent this (outside of calling python app.py in the container)?

Upvotes: 6

Views: 1173

Answers (2)

mafrosis
mafrosis

Reputation: 2770

When installing pip as editable in the current directory, the <package>.egg-info directory needs to be present for python to find the installed package.

One can “whitelist” directories from being overwritten by a docker-compose.yml mount point, like so:

services:
  myapp:
    image: mafrosis/myapp:dev
    build:
      context: .
    volumes:
      - ./:/app
      - /app/.git
      - /app/myapp.egg-info

In this example, both the .git and myapp.egg-info directories will be present when the container starts.

Upvotes: 2

timmwagener
timmwagener

Reputation: 2526

For anybody interested in calling the Python entry_points console scripts via docker-compose run during development. Copying the .egg-info folder into the mounted host directory from the running container one time works for me.

Copy the .egg-info directory to host from running container. Turn mounting off beforehand so the .egg-info directory is not overriden.

sudo docker cp <container id>:/path/to/package/in/container/package.egg-info /path/to/mounted/package/on/host/package.egg-info

Re-enable mounting, the needed .egg-info folder is now present so the installed entry_points should work and you can run commands like:

sudo docker-compose run <service name> <entry_point console script name> [OPT] [ARG]

For this to work, the Python package needed to be installed in development mode in the image, like with RUN pip install -e /path/to/package/, because otherwise the .egg-info file will not be in the symlinked source code directory, I guess.

Of course, the alternative, like @Adam mentions, is to always call the CLI scripts directly like:

sudo docker-compose run <service name> python /path/to/package/cli.py [OPT] [ARG]

Upvotes: 1

Related Questions