CODEWITHSUNDEEP
pythongitpippypi
oarfish
oarfish

Reputation: 4621

How can I use git repos as dependencies for my PyPi package?

I have a package which I'm pushing to PyPi and some of the depedencies are not packages, but installable git repositories. My requirements.txt looks like this

sphinx_bootstrap_theme>=0.6.5
matplotlib>=2.2.0
numpy>=1.15.0
sphinx>=1.7.5
sphinx-argparse>=0.2.2
tensorboardX
tqdm>=4.24.0
Cython>=0.28.5

# git repos
git+git://github.com/themightyoarfish/svcca-gpu.git

Accordingly, my setup.py has this content:

#!/usr/bin/env python

from distutils.core import setup
import setuptools
import os

with open('requirements.txt', mode='r') as f:
    requirements = f.read()
    required_pkgs, required_repos = requirements.split('# git repos')
    required_pkgs = required_pkgs.split()
    required_repos = required_repos.split()

with open('README.md') as f:
    readme = f.read()

setup(name=...
      ...
      packages=setuptools.find_packages('.', include=[...]),
      install_requires=required_pkgs,
      dependency_links=required_repos,
      zip_safe=False,   # don't install egg, but source
)

But running pip install <package> does not actually install the git dependency. I assume that pip doesn't actually use the setup script. It works when I run python setup.py install manually.

Edit:

I also tried removing dependency_links and just using install_requires with the repository, but when installing my repository from GitHub (the project including the above files), I'm met with

    Complete output from command python setup.py egg_info:
error in ikkuna setup command: 'install_requires' must be a string or 
list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'+git://g'"

It has been suggested in other answers that one can put something like

git+https://github.com/themightyoarfish/svcca-gpu.git#egg=svcca

into requirements.txt, but that fails with

   error in <pkg> setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'+https:/'

Question: (How) Can I list git repositories as dependencies for a pip package?

Upvotes: 17

Views: 4821

Answers (3)

user12520082
user12520082

Reputation: 11

We tried it with pyproject.toml and it didn't work for us either.

requires = [
    "geci_plots @ git+https://github.com/IslasGECI/[email protected]",
    "lmfit",
    "pandasql",
    "scikit-learn==1.1.3",
]

It seems that direct references don't allow it.

ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/        
         Invalid value for requires_dist. Error: Can't have direct dependency:  
         'geci_plots @ git+https://github.com/IslasGECI/[email protected]'

I assume the direct references are the GitHub repos.

Upvotes: 1

oarfish
oarfish

Reputation: 4621

Out of the 50 or so different ways to specify git dependencies for Pip, the only one that did what I intended was this one (outline in PEP 508):

svcca @ git+ssh://[email protected]/themightyoarfish/svcca-gpu

This can be used in install_requires, which solves the issue of dependency_links being ignored by pip.

An amusing side-effect is that the package cannot be uploaded to PyPi with such a dependency:

HTTPError: 400 Client Error: Invalid value for requires_dist. Error: Can't have direct dependency: 'svcca @ git+ssh://[email protected]/themightyoarfish/svcca-gpu' for url: https://upload.pypi.org/legacy/

Upvotes: 15

R. Garc&#237;a
R. Garc&#237;a

Reputation: 823

According to the next post related to How to state in requirements.txt a direct github source. You could add a package from git remote repository with the next syntax:

-e git://github.com/themightyoarfish/svcca-gpu.git

Reference: Install a project in editable mode (i.e. setuptools “develop mode”) from a local project path or a VCS url with-e

Upvotes: -1

Related Questions