CODEWITHSUNDEEP
pythonsetuptoolspyproject.toml
pandichef
pandichef

Reputation: 796

"requires" (pyproject.toml) with a local package in editable mode

I am using a pyproject.toml file to allow me to use third party packages during my build. (For example, I'd like to use the toml package in setup.up.) When I add a local package to "requires" (installed in editable mode), the build doesn't see the package. Is there a way to include local packages in my pyproject.toml other than explicitly deploying them to pypi?

Here's what my pyproject.toml file looks like currently:

[build-system]
requires = ["setuptools", "wheel", "toml", "my_local_package"]

Upvotes: 3

Views: 5357

Answers (2)

Kaia
Kaia

Reputation: 907

TL;DR update for 2024:

Ensure you editable-install your my_local_package first.

pip install --upgrade pip
pip install -e path/to/project_common
pip install -e path/to/project_that_depends_on_common

(The pip install --upgrade pip is because the ability to editable-install a pyproject.toml project is relatively recent.)

Longer explanation

The underlying thinking is, as far as I understand, that pyproject.toml isn't solely intended as a way for you to build things locally; it's also a way that to define what needs to happen when somebody pip installs your package. So a build backend can't do anything with something like:

Requires-Dist: ../../my_local_package/

Because when you package it, that doesn't exist! However, if you make sure to pip install -e ../path/to/my_local_package first, then it won't try to download it from indexes.

Example:

Suppose my directory structure is as follows:

git_repo/
  core/
    __init__.py
    pyproject.toml
  extra/
    __init__.py
    pyproject.toml
  requirements.txt

core/pyproject.toml:

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "project-core"
version = "0.0.1"
dependencies = [
]

[tool.setuptools.packages.find]
where = ["."]

extra/pyproject.toml:

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "project-extra"
version = "0.0.1"
dependencies = [
    'project-core'
]

[tool.setuptools.packages.find]
where = ["."]

And I do the following:

$ python -m venv .venv; .venv/Scripts/activate
(.venv) $ pip install -e ./extra
[...]
ERROR: Could not find a version that satisfies the requirement project-core (from project-extra) (from versions: none)
ERROR: No matching distribution found for project-core

But if we install them in the other order, it works.

(.venv) $ pip install -e ./core
[...]
Successfully built project-core
Installing collected packages: project-core
Successfully installed project-core-0.0.1

(.venv) $ pip install -e ./extra
Successfully built project-extra
Installing collected packages: project-extra
Successfully installed project-extra-0.0.1

(.venv) $ python
>>> import extra
>>> import core

You can put this in a requirements.txt file and install it with pip install -r requirements.txt:

-e ./core
-e ./extra

Various buildsystems have their own solutions for this. UV has workspaces, Poetry lets you specify development-mode paths.

[tool.poetry.dependencies]
my-package = {path = "../my/path", develop = true}

Upvotes: 0

johnbaltis
johnbaltis

Reputation: 1588

You can do something like:

[build-system]
build-backend = "setuptools.build_meta"
requires = [
  "setuptools >= 61.2",
  "versioningit @ file:///Users/basnijholt/Downloads/versioningit",
  "wheel",
]

Upvotes: 2

Related Questions