Behavior of __all__ in __init__ on Python when used to import modules

Question

I have a python package of the form:

package
├── __init__.py
├── module_1.py
└── module_2.py

Inside __init__.py I have:

__all__ = ["module_1", "module_2"]

Module_1 is:

__all__ = ["foo"]
def foo(): pass

Module_2 is:

__all__ = ["Bar"]
class Bar: pass

From the documentation I thought that the following code would import foo and Bar:

import package as pk

However, when I run pk.foo() it throws an error: AttributeError: module 'package' has no attribute 'foo' (same for Bar).

Thanks to this answer, I know that to get the desired behavior, I can change __init__.py into:

from .module_1 import *
from .module_2 import *

The above works.

However, I do not understand the documentation. The lines:

if a package’s __init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported when from package import * is encountered

Sound like my original __init__.py should have worked (the one with __all__ = ["module_1", "module_2"]). That is, the line import package as pk should have imported the modules module_1 and module_2, which in turn make foo and Bar available.

What am I missing?

EDIT:

I have also tried using exactly what the documentation mentions. That is, using from package import *, and then tried with package.foo() and foo(), but neither worked.

The first, package.foo(), throws the error NameError: 'package' is not defined.. The second, foo(), throws the error NameError: 'foo' is not defined..

How would a working example look like?.. One where __init__.py is of the form __all__ = ["module_1", "module_2"].

Answer 1

I will try to summarize the knowledge gained from the comments to my question, the documentation, my own tests, and this post.

1) __all__ behaves differently on __init__ and modules

1.1) Within a module

When __all__ is within a module, it determines what objects are made available when running from module import *.

Given this package structure:

package
├── __init__.py
├── module_1.py
└── module_2.py

And given the following code for module_1:

__all__ = ["foo"]
def foo(): pass
def baz(): pass

Running from package.module_1 import * will make foo available but not baz.

Moreover, foo can then be called using foo(), i.e., there is no need to reference the module.

1.2) Within __init__ (my original question)

When __all__ is within __init__,

it is taken to be the list of module names that should be imported when from package import * is encountered.

Running from package import * will then have two effects:

1. The scripts of the modules in __all__ will be ran (they are imported).
2. These modules are made available in the namespace.

This means that if __init__ is of the form:

__all__ = ["module_1", "module_2"]

Then, running from package import * will run the scripts of module_1 and module_2 and make both modules available. So, now, the function foo inside module_1 can be called as module_1.foo() instead of package.module_1.foo().

However, if this is the intent, then using from package.module_1 import foo might be better. As it makes foo accessible as foo().

2) from package import * is not the same as import package

Running from package import * has the effect mentioned in 1.2). However, this is not true for running import package: i.e. module_1.foo() would not work in this scenario.

3) The alternative approach to __init__

(The following is based on this post) As I mentioned in my question, there is an alternative approach to __init__, in which the objects that you want to make available when the user calls from package import * are directly imported into __init__.

As an example, __init__ could contain the following code:

from .module_1 import *
from .module_2 import *

Then, when the user calls from package import * the objects from modules 1 and 2 would be available on the namespace.

If module_1 was that of 1.1), then the function foo could be called without referencing the module. i.e. foo(). However, the same would not work for baz.

As mentioned in 2), from package import * is different to import package. Calling import package in this scenario (with this __init__), makes foo available through package.foo(), instead of just foo(). Similarly import package as pk makes foo available as pk.foo().

This approach could be preferable to that of 1.2), in which foo is made available through module_1.foo().

Answer 2

The key problem in your code is that you have defined __all__ in sub modules. Which allows to be exported only that module, but such module is not available in your directory. They are definitions you want to use.

So, just remove them from sub modules and you'll get it working.