CODEWITHSUNDEEP
pythonpackage
Marseille07
Marseille07

Reputation: 117

Python module layout where subdirectory name is the same as file

I am used to a scripting language where it is very common to lay out modules like so:

lib/Foo.pm
lib/Foo/Bar.pm
lib/Foo/Baz.pm

Where you have Foo.pm, then there is a directory called Foo and underneath you place submodules.

In Python, I tried to mimic the layout like so:

modules/foo.py
modules/foo/bar.py
modules/foo/baz.py

However, this wouldn't work because when I do:

from foo import Foo

Python thinks I'm importing a directory named 'foo' rather than foo.py. I tried to mess with init.py business to no avail.

Is there a way to get around this issue? I find it quite annoying that Python cannot distinguish between directory foo vs file foo.py.

EDIT: I think I was missing an important piece. I am sourcing the modules/ directory via my unit tests, so the entire directory structure looks like so:

modules/foo.py
modules/foo/bar.py
modules/foo/baz.py
tests/unit.py

In unit.py, I have the following:

#!/usr/bin/python

import sys
import os

findbin = os.path.abspath(os.path.dirname(sys.argv[0]))
sys.path.append(findbin + "/../modules")

from foo import Foo

obj = Foo()

When I run this, I get:

from foo import Foo
ImportError: cannot import name Foo

Upvotes: 1

Views: 1846

Answers (3)

Gabriel
Gabriel

Reputation: 10904

Creating packages with Python can be done using __init__.py files, but it's important to understand the import priority. The command 

from foo import Foo

will look for something named foo in sys.path and then look for something named Foo "inside" foo. For example, suppose you did not have any __init__.py files in your directory foo and your foo.py file contains, 

class Foo:
    def __init__(self):
        print 'hello'

Giving the command from foo import Foo from the module directory will work. 

>>> from foo import Foo
>>> f = Foo()
hello

If you want the directory foo to be a package (i.e. a collection of related modules) then you can put an empty __init__.py file in the foo directory. This however will make the import command ambiguous. I haven't found any documentation that claims packages have higher import priority than modules, but you can verify that Python is importing the package by doing the following, 

import foo
>>> foo.__file__
foo/__init__.py

which means the package got imported. I imagine the fact that packages have priority will not always be the case. In the end it's best to remove the name conflict by renaming either the file or the package.

Upvotes: 0

Eevee
Eevee

Reputation: 48594

You must do this:

modules/foo/__init__.py  # instead of foo.py
modules/foo/bar.py
modules/foo/baz.py

Perl's layout doesn't work; if a module has children, the root lives alongside them.

And for what it's worth, some Python devs feel strongly that you should leave __init__.py empty (or at least minimal), since it's imported first anytime you import any of the children.

It's possible you're also having some other problems if __init__.py didn't work for you, especially if you literally have a modules/ directory, but I can't diagnose without more details :)

Aha. DO NOT mess with sys.path; that's always a sign you're making life harder for yourself. (Do you have to mess with sys.path to import anyone else's code? Then why should you have to do it for your own, when it's right there?)

Just put them directly in the root of your source tree, as project.git/foo/bar.py. As long as you run your test harness (or whatever) from your root, you can import them with no problems.

You might also want to try a test library like pytest which does discovery and some various fancy things for you.

See also: Filesystem structure of a Python project

Or a smallish project of mine for a more practical example: https://github.com/eevee/dywypi

Upvotes: 1

user2963623
user2963623

Reputation: 2295

What you want to create is packages.

in the folder foo put a blank file with the name __init__.py. Python will then treat foo as a package. If modules is also to be treated as a package put a blank __init__.py inside it as well.

So from your question, if the packages are done correctly and you have this structure:

modules/__init__.py
modules/foo/__init__.py
modules/foo/bar.py
modules/foo/baz.py

You can do these:

import module
from module import foo
from module.foo import bar
import module.foo.bar as foobar

And a lot of other stuffs!

Upvotes: 5

Related Questions