CODEWITHSUNDEEP
pythonpipsetuptools
LudvigH
LudvigH

Reputation: 4737

Specifying package data in pyproject.toml

I use setuptools. In setup.cfg, I can define

[options.package_data]
myModule =
    '*.csv'

to make sure that data will be installed for my users.

Can I achive the same with pyproject.toml instead?

Upvotes: 51

Views: 47167

Answers (7)

JohnA
JohnA

Reputation: 821

One of the things I like is to be able to make a change and then double-check it before(!) I commit it.

For this issue, one thing you can do is open the src/mymod.egg-info directory. In it, there is a SOURCES.txt file and in that file your data files should show up:

    LICENSE.txt
    MANIFEST.in
    README.md
    pyproject.toml
    setup.cfg
    src/mymod/__init__.py
    src/mymod/some_cool_script1.py
    src/mymod/some_data.json       <=== a data file!
    src/mymod/some_script2.py
    <etc.>

So do this:

  • modify pyproject.toml to get the data you want.
  • DO NOT change the version!
  • run your process to build the egg. I use do_publish_setup.py:
import setuptools

setuptools.setup()

run it

python do_publish_setup.py -q sdist
  • run twine to check it:
twine check dist/*
  • if there's no issues, then the egg-info should have been created and you can double-check the SOURCES.txt file for everything you expect.

This process would be very nice to put into the setuptools doc... somewhere.

PS Here's the relevant pyproject.toml content:

[tool.setuptools.packages.find]
where = ['src']
exclude = ['.gitignore']

[tool.setuptools]
include-package-data = true  <== not sure if this is 100% needed

[tool.setuptools.package-data]
mymod=['*.json']        <== not sure if mymod should use quotes

Upvotes: 0

LudvigH
LudvigH

Reputation: 4737

Starting in setuptools version 61.0.0 there is beta support for this feature. See the documentation https://setuptools.pypa.io/en/stable/userguide/datafiles.html

The syntax is

[tool.setuptools.package-data]
myModule = ["*.csv"]

Update 2024-01-12: This is no longer in beta. A handful of different mechanisms can enable for data inclusion. One mechanism is to ask for automatic discovery based on git. It should suffice to set the setuptools version and enable the -scm plugin. Now all files tracked by git will be considered data files. The relevant section is...

[build-system]
requires = [
    "setuptools>=60",
    "setuptools-scm>=8.0"]

I produced a complete example and posted on github. https://github.com/el-hult/mypkg

Read the full documentation for more options.

Upvotes: 50

Timmmm
Timmmm

Reputation: 96547

As of now, if you are using pyproject.toml and setuptool you do not need to do anything. Just add some extra files in the same directory as your Python code.

But note that it will only add them if they are checked into source control, and you are using setuptools-scm (the example tells you to so you probably are).

For example this is my pyproject.toml

[build-system]
requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"

[project]
name = "foo"
authors = [
    {name = "...", email = "..."},
]
description = "..."
readme = "README.md"
requires-python = ">=3.9"
dependencies = [
    "pydantic",
    "jinja2"
]
version = "1.0.1"

[project.scripts]
foo = "foo:main"

And I have these files:

.
├── pyproject.toml
├── pyrightconfig.json
├── README.md
├── setup.py
└── src
    └── foo
        ├── command.py
        ├── index.html
        ├── __init__.py
        ├── render.py
        └── triage.py

And when I run python3 -m build it does add index.html to the wheels.

Also setup.py is just for backwards compatibility. You don't actually need it.

Upvotes: 5

Larry E
Larry E

Reputation: 179

Assuming a project dir structure like

project_root_directory
├── pyproject.toml 
└── src
    └── mypkg
        ├── __init__.py
        ├── data1.rst
        ├── data2.rst
        ├── data1.txt
        └── data2.txt

simply add the following to your pyproject.toml

[tool.setuptools.packages.find]
where = ["src"]

[tool.setuptools.package-data]
mypkg = ["*.txt", "*.rst"]

More info can be found in the docs @ https://setuptools.pypa.io/en/latest/userguide/datafiles.html

Upvotes: 11

Michele Peresano
Michele Peresano

Reputation: 416

In addition to the other existing answers, one can also simply use a MANIFEST.in file.

See https://packaging.python.org/en/latest/guides/using-manifest-in/

Upvotes: 4

brandonscript
brandonscript

Reputation: 72855

If you're using Hatchling, per the docs, you can also do this via:

[tool.hatch.build]
include = [
  "myModule/**/*.csv",
  "**/*.py",  
]
exclude = [
  "tests/**",
]

Upvotes: 2

cizario
cizario

Reputation: 4269

if I understand your concern, you are using setuptools as a building and distributing system and you want to move some configs from setup.[py,cfg] namely package_data to pyproject.toml, if so you have to use an other tool to build and distribute your package e.g poetry as stated in @Romibuzi's answer because it's not possible unless the setuptools' team plan a new major release to include a full support of pyproject.toml and then no need for extra/standalone config setup.cfg file.

some references:

Update

as per v61.0.0, setuptools brings a partial support (still in beta version) for this feature.

refer to @LudvigH's answer. (thank you btw)

Upvotes: 6

Related Questions