Correct Python project structure for packages

Question

I am creating a Python utility that will need to be shared internally via pip packages, and will be consumed both as a library and as a command-line tool.

I am having difficulty with the correct use of namespaces and where/when/if to put __init__.py files to get a consistent codebase.

• I can get "library" usage working, so I can from example_service.example import Example.
• I can get CLI usage working, so I can execute python example_service/example_cli.py.
• I can get pytest working.
• I CANNOT get all three working at once.

So

• Should my package's use from filename import Class or from .filename or from example_service.filename?
• Should I put a __init__.py in my package directory or not? It seems it should not be necessary for Python3, but I get mixed results.

---

• If
  • example.py uses from componenta import ComponentA and
  • example_cli.py uses from example import Example
  • then the CLI invokation works
  • the library import fails with no component named 'componenta'
• If
  • example.py uses from example_service.componenta import ComponentA and
  • example_cli.py uses from example import Example
  • then the CLI invokation fails with no component named 'example_service'
  • the library import works

---

I currently have a project directory that looks something like this:

project_root/
    example_service/
        example.py
        example_cli.py
        componenta.py
        componentb.py
        tests/
            __init__.py
            test_example.py
    setup.py
    requirements.txt

Inside my main lib code, example.py, I have:

from componenta import ComponentA

class Example(object):
    def foo(self):
        a = ComponentA()
        print("Example.foo()")

My CLI wrapper will handle args and stuff, but for now is just:

#!/usr/bin/env python
from example import Example

class ExampleCli(object):
    def __init__(self):
        print("ExampleCli.init()")
        e = Example()
        e.foo()

if __name__ == '__main__':
    ExampleCli()

And componenta.py is:

class ComponentA(object):
    def bar(self):
      print("ComponentA.bar()")

(Full code has been posted to GitHub)

Answer 1

A couple of points:

• It's true that a package without __init__.py should work (since Python 3.3 I think). You can leave it out, if you don't need it (i.e.: you have nothing to write in it).

• I would not call python example_service/example_cli.py directly, ever. You have the setuptools entry point for this purpose, so in your case call example directly instead. (You can remove the shebang and the executable bit from example_service/example_cli.py.)

• from within your package example_service you can either use from example_service.componenta import ComponentA or from .componenta import ComponentA. This should work the same in both cases.