CODEWITHSUNDEEP
pythonsetuptoolspyproject.toml
Abstracted
Abstracted

Reputation: 441

pyproject.toml/setuptools duplicates files into root site-packages directory

I have a problem with how pip/setuptools is installing my package. When installing from the project directory (i.e. pip install .) my project's sub-packages are duplicated and placed in the root site-packages directory. The configuration is set entirely within pyproject.toml (with a minimal setup.py for compiling a single extension).

If my package is named mypackage which contains 3 sub-packages and depends on 3 dependencies, this is the expected directory structure under site-packages in the venv:

site-packages
    - dependency1
    - dependency2
    - dependency3
    - myproject
        - subpackage1
        - subpackage2
        - subpackage3

Yet below is what I end up with, it looks like any folder containing any .py files are copied to root site-packages (i.e. including the venv itself and docs since they contain py files:

site-packages
    - dependency1
    - dependency2
    - dependency3
    - mypackage
        - subpackage1
        - subpackage2
        - subpackage3
    - subpackage1
    - subpackage2
    - subpackage3
    - docs
    - venv

What can I do to avoid duplicating sub-packages into the top-level site-packages directory/install correctly?

Here is my project structure:

myproject/
    - pyproject.toml
    - setup.py
    - docs/
    - myproject/
        - __init__.py
        - subpackage1/
        - subpackage2/
        - subpackage3/
    - venv/

The reduced contents of pyproject.toml

[project]
name = "myproject"
requires-python = ">= 3.7"
dependencies = [
    "dependency1",
    "dependency2",
    "dependency3",
]

[tool.setuptools]
packages = [
    "myproject",
    "myproject.subpackage1",
    "myproject.subpackage2",
    "myproject.subpackage3",
    ]

[build-system]
requires = ["setuptools >= 61.0.0", "cython"]
build-backend = "setuptools.build_meta"

The contents of setup.py:

from setuptools import Extension, setup
from Cython.Build import cythonize


ext_modules = [
    Extension(
        "subpackage1.func",
        ["..."],
        extra_compile_args=['-fopenmp'],
        extra_link_args=['-fopenmp'],
    )
]


setup(ext_modules=cythonize(ext_modules))

Upvotes: 3

Views: 343

Answers (1)

llamaduck
llamaduck

Reputation: 11

I've just ran into the same issue.

In my case, the build directory used by pip was polluted with the "subfolders", probably because of a previous run where my package discovery settings were erroneous. Because of this, although my configuration was (now) correct, these orphaned directories were copied to my site-packages as well.

In my case the build directory was in the folder where I called pip install . from.

If you want to find the build directory, or just check whether this is the problem, log pip's output to a file with pip install . --log foo.txt, and search for copying inside. You should see a line like:

Arguments: ('copying', '<build directory>\\lib\\subpackage1\\bar.py', ...

Hope this helps!

Upvotes: 1

Related Questions