CODEWITHSUNDEEP
pythonpypi
Windel
Windel

Reputation: 759

Does PyPI have simple urls for package downloads?

Does PyPI support simple download urls? The reason I want to do this, is that I have a PC with curl installed, but not pip. So I would be able to install the package with:

pip install ppci

But since pip is not available, what I want to do is download this package with curl and untar it.

Now I can do this:

curl https://pypi.python.org/packages/4c/e8/fd7241885330ace50d2f7598a2652d4e80c1d922faece7bba88529cf6cfe/ppci-0.5.4.tar.gz
tar xfz ppci-0.5.4.tar.gz

But what I want is a cleaner url, like this:

curl https://pypi.python.org/packages/ppci/0.5.4/ppci-0.5.4.tar.gz

So, that in future I can easily upgrade the version to this:

curl https://pypi.python.org/packages/ppci/0.5.5/ppci-0.5.5.tar.gz

Does this url, or something alike exist, such that I can easily increase the version number and get the newer version without the long hashcode in it?

Upvotes: 14

Views: 8015

Answers (4)

Cornelius Roemer
Cornelius Roemer

Reputation: 8326

Beware that since 2023 for tools that are packaged with setuptools >=v69.0.3, for package names that contain hyphens, you need to convert the hyphen to an underscore, but only for the filename part (i.e. the second time the package name is mentioned).

Example: For a package called nextstrain-cli the correct URL is:

https://files.pythonhosted.org/packages/source/n/nextstrain-cli/nextstrain_cli-8.5.0.tar.gz

Note the hyphen in the package name is converted to a hyphen in the filename. If you don't do this replacement you will get HTTP 404 NOT FOUND.

So the general rule is now:

A := first character of package name (e.g. `n`)
B := package name (e.g. `nextstrain-cli`)
C := package name with hyphen replaced by underscore (e.g. `nextstrain_cli`)
D := version specifier (e.g. 8.5.0)

https://pypi.io/packages/source/A/B/C-D.tar.gz

Reading the documentation of Pypi, it seems that this hyphen-to-underscore replacement is a general rule that applies to all non alphanumeric characters, not just hyphens:

Each component of the filename is escaped by replacing runs of non-alphanumeric characters with an underscore _:

re.sub("[^\w\d.]+", "_", distribution, re.UNICODE)

I haven't tested this, please let me know in comments if my guess is correct.

Here is a link to docs: https://peps.python.org/pep-0491/#escaping-and-unicode

Update: It turns out this stuff is even more messy than I thought.

It appears that the first full string is the package name. And the second is the sdist filename.

The filename is determined by the build tool used by the person doing the packaging. And different build tools use different rules to turn package name into filename.

So it's possible that a build tool used by nextstrain-cli changed, rather than Pypi changing anything in late 2023.

See these posts for more details:

Indeed what happened here is that sdist filenames should have always created filenames that replace hyphens with underscores. But due to a bug that was fixed only in late 2023, v69.0.3, the underscore was turned back into a hyphen. So it looked like one could just use the name for both parts of the URL. When in fact one should have always used the sdist filename for the last part.

See https://github.com/pypa/setuptools/pull/4159 for the setuptools PR.

Upvotes: 0

Windel
Windel

Reputation: 759

The right url is:

https://files.pythonhosted.org/packages/source/p/ppci/ppci-0.5.4.tar.gz

Note that this url will redirect, but curl can handle it with the -L option.

The url format is, as explained below in the comments:

https://files.pythonhosted.org/packages/source/{ package_name_first_letter }/{ package_name }/{ package_name }-{ package_version }.tar.gz

Upvotes: 20

Jack Foy
Jack Foy

Reputation: 563

These all appear to work as of 2019-10-30, and redirect one to the next:

https://pypi.io/packages/source/p/pip/pip-19.3.1.tar.gz
https://pypi.org/packages/source/p/pip/pip-19.3.1.tar.gz
https://files.pythonhosted.org/packages/source/p/pip/pip-19.3.1.tar.gz
https://files.pythonhosted.org/packages/ce/ea/9b445176a65ae4ba22dce1d93e4b5fe182f953df71a145f557cffaffc1bf/pip-19.3.1.tar.gz

This answer describes a way to fetch wheels using a similar index built by Debian: https://stackoverflow.com/a/53176862/881629

PyPI documentation actively discourages using the conveyor service as above, as it's mostly for legacy support, and we "should generally query the index for package URLs rather than guessing". https://warehouse.readthedocs.io/api-reference/integration-guide.html#querying-pypi-for-package-urls

(Thanks to Wolfgang Kuehn for the pointer to Warehouse documentation, but note that to get a correct wheel we need to select the appropriate entry for the target platform from the urls field in the API response. We can't grab a static element from the list, as order appears to vary between packages.)

Upvotes: 9

Wolfgang Kuehn
Wolfgang Kuehn

Reputation: 12936

The url for wheels is, by example invoke

https://files.pythonhosted.org/packages/py3/i/invoke/invoke-1.6.0-py3-none-any.whl

or in general

file_name := {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
first_letter := first letter of distribution

https://files.pythonhosted.org/packages/{python tag}/{first_letter}/{distribution}/{file_name}

I don't know if this is an official contract of PyPI Warehouse. You can always query, in a RestFull manner, its JSON API like so

https://pypi.org/pypi/invoke/1.6.0/json

The download url is then at document path /urls[1]/url

Upvotes: 3

Related Questions