CODEWITHSUNDEEP
pythonpipsetuptools
Matthew Feickert
Matthew Feickert

Reputation: 897

Possible to have setuptools extras_require override requirement from install_requires?

Is it possible to have setuptools's extras_require override a requirement set in install_requires? My guess is no, as extras_require is a

...dictionary mapping names of “extras” (optional features of your project)...

and so since it is "optional", then install_requires will always take precedence. I thought I would ask just in case though.

The reason for asking this is the following use case with the following example setup.py

from setuptools import setup, find_packages
setup(
  ...
  install_requires = [
    'numpy<=1.14.5,>=1.14.0',
    ...
  ],
  extras_require = {
    ...
    'tensorflow':[
       'tensorflow>=1.10.0',
       'numpy<=1.14.5,>=1.13.3',
       'setuptools<=39.1.0',
    ]
  },
  ...
)

You have a library that has support multiple backends for computation (e.g., NumPy, TensorFlow, PyTorch) but the default backend installed is just NumPy and then different backends could be installed through different options. TensorFlow has the requirement that

tensorflow 1.10.1 has requirement numpy<=1.14.5,>=1.13.3

but you don't want to force the restraint of NumPy if the user just wants the NumPy backend. So ideally, one would have in install_requires just numpy>=1.14.0 for users that would do

pip install .

and then for users that wanted TensorFlow they would just use

pip install -e .[tensorflow]

However, this of course doesn't work, as if install_requires just has numpy>=1.14.0 then the latest PyPI release of NumPy is installed (at this time 1.15.1) and you get the warning during installation of

tensorflow 1.10.1 has requirement numpy<=1.14.5,>=1.13.3, but you'll have numpy 1.15.1 which is incompatible.

So is there anyway that I can unrestrict the NumPy release in install_requires and then have setuptools check and use the releases specified in extras_require if asked for?

For context:

$ python --version
Python 3.6.6
$ pip --version
pip 18.0 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)
$ easy_install --version
setuptools 40.0.0 from /usr/local/lib/python3.6/site-packages (Python 3.6)

Upvotes: 5

Views: 1686

Answers (1)

Matthew Feickert
Matthew Feickert

Reputation: 897

The answer (after a bit more thought) is that the outcome is desired is possible, but not by overriding the install_requires, but having a requirement in install_requires that requires the library desired (so this is pretty hacky). To illustrate using this setup.py

from setuptools import setup, find_packages
setup(
  ...
  install_requires = [
    'scipy', # scipy requires numpy, and so will get the latest release from PyPI
    ...
  ],
  extras_require = {
    ...
    'tensorflow':[
       'tensorflow>=1.10.0',
       'numpy<=1.14.5,>=1.13.3',
       'setuptools<=39.1.0',
    ]
  },
  ...
)

Now

$ pip install .
$ pip freeze | grep numpy
numpy==1.15.1
$ pip freeze | grep scipy
scipy==1.1.0

and

$ pip freeze | xargs pip uninstall -y
$ pip install .[tensorflow]
$ pip freeze | grep numpy
numpy==1.14.5
$ pip freeze | grep scipy
scipy==1.1.0

Upvotes: 2

Related Questions