CODEWITHSUNDEEP
setuptoolspython-packaging
Pux
Pux

Reputation: 557

Including a data folder in a python package using setup.cfg

I have the following setup.cfg file, a usual project I want to publish to pypi and which should be shipped with a data folder, in my case ru_core_news_sm-3.1.0. (I am using setuptools):

[metadata]
name = test-project-name
version = 0.6.0

[options]
packages = find:
python_requires = >=3.6
include_package_data = True
install_requires = 
    spacy==3.1.3
    beautifulsoup4

[options.package_data]
test_project_name/ru_core_news_sm-3.1.0 = *.*

This is my directory structure

base_project_name/
├── test_project_name
│   ├── __init__.py
│   ├── a.py
│   └── ru_core_news_sm-3.1.0
├── pyproject.toml
├── MANIFEST.in
└── setup.cfg

and I have this in my MANIFEST.in:

recursive-include test_project_name/ru_core_news_sm-3.1.0 *.*

I am testing the install by executing pip install path/to/base_project_name

However, this does not get me all files in the data folder, but only some of them, and I don't know why.

These are the files in the base folder:

before

and this is what remains when I have "test installed" it using pip to some other location (both projects use venv):

after

What could be the reason?

Upvotes: 2

Views: 2900

Answers (2)

Ewout ter Hoeven
Ewout ter Hoeven

Reputation: 351

Starting with Setuptools 62.3.0, you can now use recursive wildcards ("**") to include a (sub)directory recursively. This way you can include whole folders with all their folders and files in it.

For example, when using a pyproject.toml file, this is how you include two folders recursively:

[tool.setuptools.package-data]
"ema_workbench.examples.data" = ["**"]
"ema_workbench.examples.models" = ["**"]

But you can also only include certain file-types, in a folder and all subfolders. If you want to include all markdown (.md) files for example:

[tool.setuptools.package-data]
"ema_workbench.examples.data" = ["**/*.md"]

It should also work when using setup.py or setup.cfg.

See https://github.com/pypa/setuptools/pull/3309 for the details.

Upvotes: 3

Pux
Pux

Reputation: 557

I found the problem: You should not use *.* to include all files, but instead simply *. Because sometimes files don't have an ending, which will then get ignored. Depending on the file (MANIFEST.in):

recursive-include project_name/ru_core_news_sm-3.1.0 *

or (setup.cfg)

test_project_name/ru_core_news_sm-3.1.0 = *

Upvotes: 2

Related Questions