CODEWITHSUNDEEP
pythonpython-2.7python-3.xpypi
Josh
Josh

Reputation: 12791

Finding which packages support Python 3.x vs 2.7.x

Are there any methods for automatically finding which Python versions are supported by packages on PIP?

I am looking for something that generates a table like the one below obtained automatically from the PIP index. A manually mantained table from well-known GitHub repositories and contributors from the community would also be helpful:

           2.7.x   3.x
foo         ✓       ✗
bar         ✓       ✓
baz         ✗       ✓

Has anyone compiled a table like this before? Any thoughts on how I could automatically get this information?

Upvotes: 12

Views: 756

Answers (2)

Giampaolo Rodolà
Giampaolo Rodolà

Reputation: 13076

Just use this: https://pypi.python.org/pypi/caniusepython3/

For the sake of statistics, this is openstack list of deps (161 in total): https://github.com/openstack/requirements/blob/master/global-requirements.txt

And this is what's holding them back:

  giampaolo@UX32VD:/tmp$ caniusepython3 -r requirements.txt 
  Finding and checking dependencies ...

  You need 67 projects to transition to Python 3.
  Of those 67 projects, 65 have no direct dependencies blocking their transition:

  boto
  cmd2
  coinor.pulp
  croniter
  ddt
  diskimage-builder
  django-bootstrap-form
  django-compressor
  django-openstack-auth
  dnspython
  eventlet
  extras
  gear
  hacking
  thrift (which is blocking happybase)
  jsonrpclib
  mysql-python
  netifaces
  nose-exclude
  nosehtmloutput
  nosexcover
  openstack-doc-tools
  openstack.nose-plugin
  os-apply-config
  os-collect-config
  os-refresh-config
  oslo.config
  oslo.messaging
  oslo.rootwrap
  oslo.sphinx
  oslosphinx
  pam
  ecdsa (which is blocking paramiko)
  paste
  posix-ipc
  proboscis
  pycadf
  pyghmi
  python-ceilometerclient
  python-cinderclient
  python-designateclient
  python-glanceclient
  python-heatclient
  python-ldap
  python-neutronclient
  python-openstackclient
  python-savannaclient
  python-seamicroclient
  python-swiftclient
  python-troveclient
  qpid-python
  rtslib-fb
  sockjs-tornado
  sphinxcontrib-docbookrestapi
  sphinxcontrib-httpdomain
  sphinxcontrib-pecanwsme
  sqlalchemy-migrate
  suds
  swift
  taskflow
  tripleo-image-elements
  warlock
  websockify
  xenapi
  zake

Upvotes: 4

vgel
vgel

Reputation: 3345

Unfortunately the PyPI API seems to be pretty bad. You can interact with it through XMLRPC, but the return values are poorly documented (py_version can be blank or 'source', whatever that means, for example). Probably the most accurate thing to do would be to download each package and use heuristics, but aside that, the tag system seems to be the best bet. It's not supported by the API, but you can scrape it out of the webpage pretty easily:

def package_supports_py3(pkg):
    b = bs4.BeautifulSoup(requests.get('https://pypi.python.org/pypi/' + pkg).text)
    return 'Programming Language :: Python :: 3' in map(lambda li: li.find('a').text, b.find('strong', text = 'Categories').next_sibling.next_sibling.find_all('li'))

>>> package_supports_py3('virtualenv')
True
>>> package_supports_py3('oh-my-vim')
False # but it doesn't have /any/ python version related tags, so who knows...

(requires BeautifulSoup4 and requests)

This method won't produce a false positive, but it has a good chance of missing packages who have actually upgraded to py3. If you need this to be reliable, I would suggest a combination of checking python_version, this method, and manual overrides. If you're just screwing around, this method should be good enough since most of the larger, well-run projects seem to respect the tagging guidelines.

Upvotes: 3

Related Questions