CODEWITHSUNDEEP
pythonpypi
Tiago Lubiana
Tiago Lubiana

Reputation: 339

How to put data in a PyPi python package and retrieve via pip?

I am trying to package some data alongside the scripts within a package of mine: https://pypi.org/project/taxon2wikipedia/0.0.4/

The source distribution seems to contain the files that I need, but when trying to use the package I get the error message:

FileNotFoundError: [Errno 2] No such file or directory: '~/my_venv/lib/python3.8/site-packages/taxon2wikipedia/dicts/phrase_start_dict.json'

The "data" and "dicts" folders are not there in "site-packages/taxon2wikipedia", but are present in the package when I manually download it? Any suggestions on why it might be happening?

Thanks!

Edit: extra information

I already have a MANIFEST.in file with recursive-include src *.rq *.py *.jinja *json, and even changing it to other similar options, it did not work. I am using a pyproject.toml configuration and a mock setup.py to run setuptools.setup(). I run python3 -m build to build the package. Maybe the problem lies there?

This is the source repository by the way: https://github.com/lubianat/taxon2wikipedia

The files in PyPI are all correct, but don't seem to be downloaded when I pip install the package.

Edit 2 - Related question

Why does "pip install" not include my package_data files?

It seems to be some problem with how pip installs the package. The solution is different as in my case the files seem to be at the correct directory.

Upvotes: 3

Views: 803

Answers (2)

Tiago Lubiana
Tiago Lubiana

Reputation: 339

I came across this blog post which helped me solve the issue: https://jwodder.github.io/kbits/posts/pypkg-data/ In the end, I was missing [options.package_data] * = *.json *.jinja *.rq from my setuf.cfg

Upvotes: 1

AKX
AKX

Reputation: 168967

You need a MANIFEST.in file in your source directory to tell the packaging process which non-Python files need to come along in a built package.

In your case, a MANIFEST.in file with

recursive-include taxon2wikipedia *.json

should do the trick.

Since .whl files are zips, you can use zipinfo dist/yournewwheel.whl to see that the files are there too before pushing a release.

Upvotes: 1

Related Questions