CODEWITHSUNDEEP
pythonpackaging
YTKme
YTKme

Reputation: 187

How To Properly Include Miscellaneous File With Python Source Distribution

I am unable to get miscellaneous (non-Python) files to be include in a Python project with pip install.

I tried using include_package_data = True and MANIFEST.in in setup(), but it did not include the .html files I wanted. I tried using package_data in setup() as well but same result.

The data_files in setup() works. But it doesn't copy the .html file to the site-packages folder, instead it just copy to the root of the environment with a folder name resource as specified in the configuration.

Any one have idea what I am missing here?

I have the following project structure

ProjectA
- res/
    - template_a.html
    - template_b.html
    - template_c.html
- tool/
    - __init__.py
    - modulea.py
- constant.py
- project_a.py
- ...
- MANIFEST.in
- setup.py
- ...

setup.py

from setuptools import setup, find_packages

# Read the contents of the README.md file
from pathlib import Path
current_directory = Path(__file__).parent
long_description = (current_directory/'README.md').read_text()

setup(
    name = 'project_a',
    version = '0.0.1',

    # Dependency
    install_requires = [
        'requests',
        ],

    # Metadata
    author = '',
    author_email = '',
    description = 'Project A.',
    long_description=long_description,
    long_description_content_type='text/markdown',
    packages = [
        'tool',
        ],
    py_modules = [
        'project_a',
        'constant',
        '...'
        ],
    # data_files = [
    #     ('resource', [
    #         'res/template_a.html',
    #         'res/template_b.html',
    #         'res/template_c.html',
    #         ]),
    #     ],
    python_requires = '>=3.7',
    # include_package_data = True,
    # package_data = {
    #     '': ['res/*.html']
    #     },
    )

MANIFEST.in

include res/*.html

Upvotes: 0

Views: 699

Answers (1)

metatoaster
metatoaster

Reputation: 18908

The first issue is that res is not declared as part of packages - package_data only works for entities that have been included as part of packages.

The other issue is that declaration of package_data has an empty string as the key - it doesn't actually reference the root directory but package directories, and only for file paths at that. It does not correspond the root directory of the site-packages (or dist-packages).

The MANIFEST.in file and data_files declaration in setup.py is not part of the required solution, and thus can be omitted.

To fix the issues, you will need to declare res as part of packages, and that package_data will need to reference that "package" as the root (to limit the include files) and a list of desired files (in this case, all the html files via the *.html glob syntax) to be part of the installation. The corrected setup.py will need to include the changes to ensure python setup.py bdist_wheel include the files you wish to include into the package:

setup(
    name = 'project_a',
    # ...
    packages = [
        'tool',
        'res',  # add this
    ],
    # ...
    include_package_data = True,
    package_data = {
        'res': ['*.html'],   # include this; pairs with the entry in `packages`
        # alternatively, uncomment the following to include any
        # *.html files from any of the `packages` declared above, 
        # e.g tools/*.html and res/*.html will be included as tool 
        # and res are inside `packages`.
        # '': ['*.html'],
    },
)

Running python setup.py bdist_wheel should now produce:

$ python setup.py bdist_wheel
running bdist_wheel
...
writing manifest file 'project_a.egg-info/SOURCES.txt'
creating build/lib/res
copying res/template_a.html -> build/lib/res
copying res/template_b.html -> build/lib/res
copying res/template_c.html -> build/lib/res
...
copying build/lib/tool/modulea.py -> build/bdist.linux-x86_64/wheel/tool
copying build/lib/project_a.py -> build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/res
copying build/lib/res/template_c.html -> build/bdist.linux-x86_64/wheel/res
copying build/lib/res/template_a.html -> build/bdist.linux-x86_64/wheel/res
copying build/lib/res/template_b.html -> build/bdist.linux-x86_64/wheel/res
copying build/lib/constant.py -> build/bdist.linux-x86_64/wheel
...
creating 'dist/project_a-0.0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'constant.py'
adding 'project_a.py'
adding 'res/template_a.html'
adding 'res/template_b.html'
adding 'res/template_c.html'
adding 'tool/__init__.py'
adding 'tool/modulea.py'
adding 'project_a-0.0.1.dist-info/METADATA'
adding 'project_a-0.0.1.dist-info/WHEEL'
adding 'project_a-0.0.1.dist-info/top_level.txt'
adding 'project_a-0.0.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel

Note that the additional output compared to the version without the 'res' entries in packages and package_data have been bolded. The resulting .whl file may then be found in dist and it may be used for installation (e.g. via pip install -U dist/project_a-0.0.1-py3-none-any.whl).

Upvotes: 1

Related Questions