CODEWITHSUNDEEP
pythonresourcesvirtualenvpython-wheel
vpa
vpa

Reputation: 401

how to access resource files after pip install in virtual env in python?

Let says that I have this project structure:

src
  |my_package
     __init__.py
    |utils
      __init__.py
      util.py
    |resources
      __init__.py
      my_resource.yml

In util.py, I have this code which need the resource file to work:

import yaml
import importlib.resources
from my_package import resources

class Util:

  def merge_settings(self, settings: dict)->dict:
    with importlib.resources.path(resources, 'my_resource.yml') as p:
        with open(p) as file:
            default_settings = yaml.safe_load(file)

and everything works fine in my development environment.

Then I make a wheel with this code with my setup.py file:

import setuptools
import glob

resource_folder = 'my_package/resources'

setuptools.setup(
  name="my_package", 
  version="0.3",

  packages=setuptools.find_packages(),
  data_files=[(resource_folder, glob.glob(resource_folder+r'/*.yml'))]

then I create a wheel:

python .\setup.py bdist_wheel

and I finally install it to be used in another project, using a virtual environment with name my_env:

(my_env) D:\dev pip install my_package-0.3-py3-none-any.whl

But my code is not running any more due to this line:

importlib.resources.path(resources, 'my_resource.yml')

The reason is found when exploring my_env folder, my_resource.yml is not in my_package anymore.

my_env
  |my_package
    |resources
      my_resource.yml
  |Lib
    |site-packages
      |my_package
        |resources
          __init__.py

But this location could be quite useful to modify easily this file... The how can I deal in the same time with a correct call of resources in my development environment and when using it after pip install ? I would like to always have access to the yml file for edition when required, even after pip install...

Tks for your help

Upvotes: 1

Views: 989

Answers (1)

anthony sottile
anthony sottile

Reputation: 70175

your data_files is both mis-specified and not the setting you want (it's intended for non-package data). the keys in data_files are placed from the root of the prefix (so say you install your package into ./venv instead of your data ending at ./venv/lib/python#.#/site-packages/my_package/resources/... they're going to end up at venv/my_package/resources -- definitely not what you want!).

the actual setting that you want is package_data:

    package_data={
        'my_package.resources': '*.yml',
    },

the mapping maps from dotted package names to globs and will place it inside site-packages

there's no need to use MANIFEST.in, etc. as these files are automatically included in your package

for more on this, I made a video on the subject

Upvotes: 3

Related Questions