CODEWITHSUNDEEP
pythonpytest
KirkD-CO
KirkD-CO

Reputation: 1793

__init.py__ and pytest - cannot import base modules when __init__.py present

I have a Python project in which I have code organized as follows:

/package
   LICENSE
   README.md
   setup.py
   /project
      /mod_1
         mod_1.py
         __init__.py
         /tests
            test_mod_1.py
      /mod_2
         mod_2.py
         __init__.py
         /tests
            test_mod_2.py
      object_factory.py
      __init__.py

The object_factory.py file contains a class that is used to build objects defined in mod_1.py and mod_2.py.

Within the tests, each test file imports the class from object_factory.py. When I run pytest from the /project directory, it complains that the object_factory module does not exist. If, however, I remove __init__.py from each of the mod_# directories, it works fine.

I've read many blog posts and Python help posts describing the intent of __init__.py as well as pytest, but I cannot seem to sort out this one detail.

Upvotes: 1

Views: 2045

Answers (1)

andylvua
andylvua

Reputation: 352

1. __init__.py files

I've read many blog posts and Python help posts describing the intent of init.py as well as pytest, but I cannot seem to sort out this one detail.

Answering your question: __init__.py files are used to mark your directory as a Python package. This means that Python will look for submodules inside that directory, so you can import them.

Alongside, pytest requires you to write tests for your package in a separate packages. It means that you should consider putting __init__.py in your tests directory.

2. Project structure

Talking about project structure, pytest provides two basic test layouts in their Good integration practices

The first one is Tests outside application code

pyproject.toml
setup.cfg
mypkg/
    __init__.py
    app.py
    view.py
tests/
    test_app.py
    test_view.py
    ...

as documentation says, this approach has a bunch of benefits:

  • Your tests can run against an installed version after executing pip install ..
  • Your tests can run against the local copy with an editable install after executing pip install --editable ..

The second one is Tests as part of application code

pyproject.toml
setup.cfg
mypkg/
    __init__.py
    app.py
    view.py
    test/
        __init__.py
        test_app.py
        test_view.py
        ...

This is actually your case.

Turning back to your project structure, here is a couple of things I want to highlight:

  1. As I've said before, consider making your test directories packages.
  2. Take a look at PEP 423 - Avoiding deep nesting. You probably don't want to complicate your package, therefore, don’t put your package modules in subdirectories unless you really need to.

Taking into account all the above, I would recommend you change your structure a little bit, so it looks like the following:

/package
   LICENSE
   README.md
   setup.py
   /project
      mod_1.py
      mod_2.py
      /tests
         __init__.py
         test_mod_1.py
         test_mod_2.py
      object_factory.py
      __init__.py

In this case, you should run you tests with the command

pytest --pyargs project

Note that you have to import your modules into object_factory.py using from project import test_mod_X

However, you probably don’t even want to distribute your test along with your application.

If I'm right, you can also use the first approach:

/package
   LICENSE
   README.md
   setup.py
   /project
      __init__.py
      mod_1.py
      mod_2.py
      object_factory.py
   /tests
      __init__.py
      test_mod_1.py
      test_mod_2.py

and run your test with pytest. It seems more readable and straightforward to me.

You may also find it useful to look at Invoking pytest versus python -m pytest

One more thing

There is one more thing I would like to add. It's something that isn't directly related to your problem but can cause confusion and, as a result, other problems later on when your code is more complex.

You may have confused the names project and package.

The project is a folder in which all your files are located - README, LICENSE, setup.py, etc.

The package is the directory where __init__.py, your code, and modules reside.

With that in mind, swap these names around to make your code more readable for yourself and others.

Upvotes: 3

Related Questions