Python modules import fails

Question

I checked similar questions, but nothing really answered my issue. Similar to another post, my package structure is:

mypackage/  
    __init__.py
    mymodule.py
    utils/ 
        __init__.py
        common.py
        myutils1.py
    data_classes/ 
        __init__.py
        command_file.py
        myclass.py

I need to use this package in multiple ways: 1) import the package and use it in larger scripts 2) run some modules as standalone (eg: python3 myutils1.py )

myutils uses functions from common, to do so, I tried to have

A)

from PySfmc.utils import common #directly in myutils.py 

B)

from mypackage import utils         # in __init__.py
from mypackage.utils import common # in myutils.py 

C)

 import common #directly in myutils.py 

etc

Long story short... I didn't manage to have something that succesfully imports both when importing in the package at the repl AND when running python3 myutils.py

anyone has an idea what I am doing wrong?

Thanks!!

---

edit, I guess what it boils down to is that: from repl I can do either of these:

from mypackage.folder.file import function
from mypackage.folder import file
from mypackage import folder

but if I write the same inside myutils1.py it will fail

---

edit2:

I edited setup.py from the following line:

packages=['mypackage'],

to

packages=['mypackage','mypackage.data_classes','mypackage.utils']

this seems to fix my problem: now everything works including the imports that were failing before.

is there any good reason why I should not do this? it seems less hacky than other suggested solutions.

Answer 1

Usually, scripts are not located within the package itself but in a distinct directory (most often named "bin") at the same level as the project package directory itself, ie:

myproject/
    bin/
        myutils1.py
        otherscript.py
    mypackage/  
        __init__.py
        mymodule.py
        utils/ 
            __init__.py
            common.py
        data_classes/ 
            __init__.py
            command_file.py
            myclass.py

This way the script only use absolute imports (from mypackage.somemodule import whatever), and relies on mypackage being available in sys.path - which is done by making your package properly installable, and allows you to have the scripts installed somewhere in your PATH.

This of course only works properly with a proper install - if you try to run python bin/myutils1.py from within myproject, the package won't still be found. This can be solved by inserting the proper path (the path to myproject at the beginning of sys.path:

#  bin/myutils1.py
import sys
import os
__PROJECT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if __PROJECT_DIR not in sys.path:
    sys.path.insert(0, __PROJECT_DIR)

from mypackage.utils import common

But you only want to do this if you're not planning on doing a properly installable package (if your package is properly installed messing with sys.path is not only useless but even possibly harmful) - and then, it's even simpler to just move your scripts directly at the top of your project dir (just above your package dir) so the python runtime will already have set sys.path correctly:

myproject/
    myutils1.py
    otherscript.py
    mypackage/  
        __init__.py
        mymodule.py
        utils/ 
            __init__.py
            common.py
        data_classes/ 
            __init__.py
            command_file.py
            myclass.py

Answer 2

If you want to run it as a package and as a standalone module, you can add this to myutils.py, it's hacky but it will work:

import os
import sys
_package_path = os.path.dirname(
    os.path.dirname(
        os.path.abspath(__file__)
    )
)
sys.path.insert(0, _package_path)

In myutils1.py you can import a_function from common.py like this:

from utils.common import a_function

In mymodule.py you can import b_function from myutils1.py like this:

from utils.myutils1 import b_function

Answer 3

If you want to run your package standalone, you need to create a starter outside the package.

starter.py
mypackage/  
    __init__.py
    mymodule.py
    utils/ 
        __init__.py
        common.py
        myutils1.py
    data_classes/ 
        __init__.py
        command_file.py
        myclass.py

starter.py should have this Code:

import mypackage.utils.myutils.py

If you run starter.py, it will work.

You can find a real world example in one of my programs. This program can be run directly with running jdTextEdit.py, which just imports the jdTextEdit package and it can be installed as module with the setup.py too. I hope, this is whaz yopu are looking for.