CODEWITHSUNDEEP
python-3.xpython-importpython-packaging
Daniel Long
Daniel Long

Reputation: 1647

Importing internal modules for a python package

I know that there's likely a duplicate of this question floating around, so I apologize ahead of time as I've spent significant time digging and I haven't been able to find it.

I'm attempting to make my python package, svmksalesforce, installable so I can share it with my coworkers. I'm struggling to set-up my internal imports so that they work both when the package is installed and when I'm executing package modules directly while building/testing the package.

For example, in order to subclass my Insert to create a BulkInsert subclass, while building the package, in my bulk_insert.py file, I have the following import statement: from insert import Insert. This executes fine when executing bulk_insert.py (__name__ == '__main__').

However, after installing my package, I receive the following error when I execute import svmksalesforce:

    from insert import Insert
ModuleNotFoundError: No module named 'insert'

After doing some research, I determined that I needed to instead import from the top-level parent directory, so I updated my import in bulk_query.py to: from svmksalesforce.insert import Insert.

This import structure allows me to run import svmksalesforce and has allowed me to share the package with my coworkers, but now when I'm adding new functionality, I can no longer execute bulk_query.py directly. I again, receive an import error: 

    from svmksalesforce.insert import Insert
ModuleNotFoundError: No module named 'svmksalesforce'

I found this post that explains why the import fails, but I can't seem to find a solution that works both when __name__ == '__main__' (executed directly) and when __name__ == 'svmksalesforce' (executed via import)

Right now I'm updating all my import statements when I need to add new functionality and then updating them all back when I need to create a new sdist. Every time I create one more commit mentioning reverting import statements, I die a little inside.

Would be super appreciative to anyone who can save me that pain.

Directory Structure:

svmksalesforce directory structure

Upvotes: 1

Views: 4008

Answers (1)

Daniel Long
Daniel Long

Reputation: 1647

I ended up finding a solution that appears to be working.

  1. Create a virtual environment to use when executing modules in your package directly
  2. Activate your virtual environment
  3. Run pip install -e ~/path/to/package. In this example, your setup.py file should be in the package folder.

This process installs the package to your virtual environment in editable mode. This means any changes you make to the modules in the package are immediately reflected in the installed version of your package in your virtual env, just as they would be if you were building and executing an internal module directly.

This allows you to then import as if the package directory is always the top-level directory. In my example in the question description, I ended up using from svmksalesforce.insert import Insert

Upvotes: 1

Related Questions