cppyy fails to install

Question

I am trying to install cppyy inside an Ubuntu 18.04 docker container, using a Python 3.6 environment. It works fine on a local (K)ubuntu 18.04 installation. It fails with an error about legacy build for wheel that produced no files.

I have updated g++ inside the Docker container to make sure that that isn't the problem.

I have updated setuptools and wheel as well.

I've also tried PIP_NO_BINARY=cppyy to make sure pipenv is not using the prebuilt wheel for cppyy.

FROM ubuntu:18.04

ENV TZ=Europe/Amsterdam
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN  export LC_ALL=C.UTF-8 && export LANG=C.UTF-8
RUN apt-get update && apt-get install -y python3.6
RUN apt-get install -y python3-pip

# Make sure proper symlinks are in place
RUN if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [ ! -e /usr/bin/python ]; then ln -sf /usr/bin/python3 /usr/bin/python; fi

RUN /usr/bin/pip install --upgrade pipenv setuptools wheel

RUN mkdir /config
COPY ./config /config
COPY ./backend /app

WORKDIR /app
RUN mkdir -p uploads

# Otherwise Pip starts whining about ASCII
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8

RUN apt-get install -y mysql-server libmysqlclient-dev

# Install Python deps
RUN pipenv install --clear --system --python=3.6

The last line fails.

The Pipfile is as follows:

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = false
name = "pypi"

[global]
no-cache-dir = false

[packages]
pandas = "*"
numpy = "*"
seaborn = "*"
matplotlib = "*"
"jinja2" = "*"
imgkit = "*"
wkhtmltopdf = "*"
bokeh = "*"
flask = "*"
flask-socketio = "*"
eventlet = "*"
flask-cors = "*"
sqlalchemy = "*"
pyopenssl = "*"
gunicorn = "*"
xlrd = "*"
mysqlclient = "*"
sqlalchemy-utils = "*"
python-dateutil = "*"
pep8 = "*"
chardet = "*"
openpyxl = "*"
cppyy = "*"

[dev-packages]
pylint = "*"
python-dotenv = "*"
mysqlclient = "*"
matplotlib = "*"

and the relevant part of Pipfile.lock is as follows:

       "cppyy": {
            "hashes": [
                "sha256:af7774eb8f7da934790b08e5e1b2b1ea514b3d1e0d07d71c3b73f6312389a473"
            ],
            "index": "pypi",
            "version": "==1.4.9"
        },
        "cppyy-backend": {
            "hashes": [
                "sha256:3cbfb92531cce600cbd8c469bc633d00dfc7d5e3d3a8b22b1596a8db9f8e9808"
            ],
            "version": "==1.8.2"
        },
        "cppyy-cling": {
            "hashes": [
                "sha256:064644d15ea6a9d02ba613696c567b58c85343b0b78ac035af3bd1295cf5887a",
                "sha256:3911b9ec8697c3fa43f6519e0b2c8006b25d2696d6a72aa46179783078b0fb20",
                "sha256:49f02c76faaf0c882777f321ac2bdda375c6deff62df8bcf3b1f6a34ddf16697",
                "sha256:7eaedb57eea1007569b8ecc3528414442ff0fb3df32a3afc57258ac3eeca0dc4",
                "sha256:a0a36074536be9b945e23b6e607ceb59b489d2ef38fe99547e7afb6b6dc78029"
            ],
            "version": "==6.15.2.7"
        },
        "cpycppyy": {
            "hashes": [
                "sha256:f2386f34499ab567e02456e5238b7d6a12883a57b632c3bbb8257058194fd19c"
            ],
            "version": "==1.7.2"
        },

The expected result is obviously "Installation succeeded" from pipenv. The actual error message is as follows:

Installing dependencies from Pipfile.lock (57997e)…
An error occurred while installing cppyy==1.4.9 --hash=sha256:af7774eb8f7da934790b08e5e1b2b1ea514b3d1e0d07d71c3b73f6312389a473! Will try again.
An error occurred while installing cpycppyy==1.7.2 --hash=sha256:f2386f34499ab567e02456e5238b7d6a12883a57b632c3bbb8257058194fd19c! Will try again.
Installing initially failed dependencies…
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/cli/command.py", line 254, in install
[pipenv.exceptions.InstallError]:       editable_packages=state.installstate.editables,
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 1874, in do_install
[pipenv.exceptions.InstallError]:       keep_outdated=keep_outdated
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 1253, in do_init
[pipenv.exceptions.InstallError]:       pypi_mirror=pypi_mirror,
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 862, in do_install_dependencies
[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, False, failed_deps_queue, retry=False)
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.6/dist-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Looking in indexes: https://pypi.python.org/simple', 'Collecting cppyy==1.4.9 (from -r /tmp/pipenv-o4uo_7oz-requirements/pipenv-mt1pg8se-requirement.txt (line 1))', '  Using cached https://files.pythonhosted.org/packages/e2/bf/46ab78ea7b5d804832cc6de4ae0d38b4543d334d323028cc6e2df4b6ae04/cppyy-1.4.9.tar.gz', 'Building wheels for collected packages: cppyy', '  Building wheel for cppyy (setup.py): started', "  Building wheel for cppyy (setup.py): finished with status 'done'", '  Running setup.py clean for cppyy', 'Failed to build cppyy', 'Installing collected packages: cppyy', '  Running setup.py install for cppyy: started', "    Running setup.py install for cppyy: finished with status 'error'"]
[pipenv.exceptions.InstallError]: ["WARNING: Legacy build of wheel for 'cppyy' created no files.", '  Command arguments: /usr/bin/python3 -u -c \'import setuptools, tokenize;__file__=\'"\'"\'/tmp/pip-install-41clln6n/cppyy/setup.py\'"\'"\';f=getattr(tokenize, \'"\'"\'open\'"\'"\', open)(__file__);code=f.read().replace(\'"\'"\'\\r\\n\'"\'"\', \'"\'"\'\\n\'"\'"\');f.close();exec(compile(code, __file__, \'"\'"\'exec\'"\'"\'))\' bdist_wheel -d /tmp/pip-wheel-a3go9c51 --python-tag cp36', '  Command output: [use --verbose to show]', '    ERROR: Complete output from command /usr/bin/python3 -u -c \'import setuptools, tokenize;__file__=\'"\'"\'/tmp/pip-install-41clln6n/cppyy/setup.py\'"\'"\';f=getattr(tokenize, \'"\'"\'open\'"\'"\', open)(__file__);code=f.read().replace(\'"\'"\'\\r\\n\'"\'"\', \'"\'"\'\\n\'"\'"\');f.close();exec(compile(code, __file__, \'"\'"\'exec\'"\'"\'))\' install --record /tmp/pip-record-77wl9so6/install-record.txt --single-version-externally-managed --compile:', '    ERROR: No eggs found in /tmp/easy_install-0dvwvzxr/CPyCppyy-1.7.2/egg-dist-tmp-94jvpfxp (setup script problem?)', '    Traceback (most recent call last):', '      File "<string>", line 1, in <module>', '      File "/tmp/pip-install-41clln6n/cppyy/setup.py", line 166, in <module>', '        zip_safe=False,', '      File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 144, in setup', '        _install_setup_requires(attrs)', '      File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 139, in _install_setup_requires', '        dist.fetch_build_eggs(dist.setup_requires)', '      File "/usr/local/lib/python3.6/dist-packages/setuptools/dist.py", line 717, in fetch_build_eggs', '        replace_conflicting=True,', '      File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 786, in resolve', '        raise DistributionNotFound(req, requirers)', "    pkg_resources.DistributionNotFound: The 'CPyCppyy>=1.7.2' distribution was not found and is required by the application", '    ----------------------------------------', 'ERROR: Command "/usr/bin/python3 -u -c \'import setuptools, tokenize;__file__=\'"\'"\'/tmp/pip-install-41clln6n/cppyy/setup.py\'"\'"\';f=getattr(tokenize, \'"\'"\'open\'"\'"\', open)(__file__);code=f.read().replace(\'"\'"\'\\r\\n\'"\'"\', \'"\'"\'\\n\'"\'"\');f.close();exec(compile(code, __file__, \'"\'"\'exec\'"\'"\'))\' install --record /tmp/pip-record-77wl9so6/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-41clln6n/cppyy/']
ERROR: ERROR: Package installation failed...

---

EDIT: the following actually works:

RUN pipenv install --clear --system --python=3.6; exit 0
RUN pipenv install --clear --system --python=3.6

the first install fails but the second succeeds without any trouble. Maybe this helps.

Answer 1

Since it succeeds the second time around, my best guess is that packages are not installed in the intended order. Or not installed completely before the next package is processed.

Based on the log file snippet, it's not clear what the attempted installation order was. The error message shown seems to be for cppyy only, but all it says is that if fails b/c it can't find eggs for CPyCppyy, so the relevant error message should come from that one. Note that building of wheels and eggs is disabled, because pip does not resolve dependencies before attempting to build those. Meaning that such builds would either fail, or use an old install if there happens to be one (which is even worse).

Anyway, the order has to be cppyy-cling, cppyy-backend, CPyCppyy, cppyy. (The reasoning behind the split is explained here: https://cppyy.readthedocs.io/en/latest/packages.html#package-structure.) Each package needs to be fully installed before the next can proceed: the cling-config script (provided by cppyy-cling) is used to propagate compiler flags (in particular it propagates which C++ standard should be supported), so that needs to be available.