CODEWITHSUNDEEP
pythondockernumpyscipypip
Hubro
Hubro

Reputation: 59313

Why doesn't pip inside my Docker container install pre-compiled packages?

Installing scipy on my host machine:

❯ pip install scipy
Collecting scipy
  Downloading scipy-1.5.4-cp38-cp38-manylinux1_x86_64.whl (25.8 MB)
     |████████████████████████████████| 25.8 MB 2.4 MB/s 
Requirement already satisfied: numpy>=1.14.5 in /home/tomas/.pyenv/versions/3.8.2/envs/3.8.2@appdaemon/lib/python3.8/site-packages (from scipy) (1.19.4)
Installing collected packages: scipy
Successfully installed scipy-1.5.4

Trying to install scipy in my Docker container (acockburn/appdaemon:latest):

/conf # pip install scipy
Collecting scipy
  Downloading scipy-1.5.4.tar.gz (25.2 MB)
     |████████████████████████████████| 25.2 MB 274 kB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpdj2d13lc
         cwd: /tmp/pip-install-o5ltw_ig/scipy
    Complete output (139 lines):
    lapack_opt_info:
    lapack_mkl_info:
    customize UnixCCompiler
      libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE
    
    openblas_lapack_info:
    customize UnixCCompiler
    customize UnixCCompiler
      libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE
    
    openblas_clapack_info:
    customize UnixCCompiler
    customize UnixCCompiler
      libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE
    
    flame_info:
    customize UnixCCompiler
      libraries flame not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE
    
    atlas_3_10_threads_info:
    Setting PTATLAS=ATLAS
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/lib/
    
      NOT AVAILABLE
    
    atlas_3_10_info:
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/lib/
    
      NOT AVAILABLE
    
    atlas_threads_info:
    Setting PTATLAS=ATLAS
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/
    
      NOT AVAILABLE
    
    atlas_info:
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib/
    
      NOT AVAILABLE
    
    accelerate_info:
      NOT AVAILABLE
    
    lapack_info:
    customize UnixCCompiler
      libraries lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE
    
    lapack_src_info:
      NOT AVAILABLE
    
      NOT AVAILABLE
    
    setup.py:460: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-ksceopr3'), proceeding with generating Cython sources and expanding templates
      warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "
    Running from SciPy source directory.
    /tmp/pip-build-env-4401a2iy/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      if getattr(self, '_calc_info_{}'.format(lapack))():
    /tmp/pip-build-env-4401a2iy/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      if getattr(self, '_calc_info_{}'.format(lapack))():
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in 
        main()
      File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 110, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-4401a2iy/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 161, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/tmp/pip-build-env-4401a2iy/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 253, in run_setup
        super(_BuildMetaLegacyBackend,
      File "/tmp/pip-build-env-4401a2iy/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 145, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 583, in 
        setup_package()
      File "setup.py", line 579, in setup_package
        setup(**metadata)
      File "/tmp/pip-build-env-4401a2iy/overlay/lib/python3.8/site-packages/numpy/distutils/core.py", line 137, in setup
        config = configuration()
      File "setup.py", line 477, in configuration
        raise NotFoundError(msg)
    numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpdj2d13lc Check the logs for full command output.

The same thing happens with numpy.

My Docker container and host machine use the exact same CPython version (3.8.2) and are both x86_64. How come the pre-compiled library isn't installed both places? Is this something I can control?

Upvotes: 2

Views: 976

Answers (1)

Hubro
Hubro

Reputation: 59313

Turns out Docker had nothing to do with it, but rather the OS running inside it. The Docker image I'm using is based on Alpine, which uses musl libc rather than glibc. All Python pre-compiled libraries are compiled for glibc, at least for now, so Alpine users are out of luck.

https://github.com/pypa/manylinux/issues/37

Upvotes: 4

Related Questions