CODEWITHSUNDEEP
python-3.xpytest
Claude
Claude

Reputation: 9980

restart python (or reload modules) in py.test tests

I have a (python3) package that has completely different behaviour depending on how it's init()ed (perhaps not the best design, but rewriting is not an option). The module can only be init()ed once, a second time gives an error. I want to test this package (both behaviours) using py.test.

Note: the nature of the package makes the two behaviours mutually exclusive, there is no possible reason to ever want both in a singular program.

I have serveral test_xxx.py modules in my test directory. Each module will init the package in the way in needs (using fixtures). Since py.test starts the python interpreter once, running all test-modules in one py.test run fails.

Monkey-patching the package to allow a second init() is not something I want to do, since there is internal caching etc that might result in unexplained behaviour.

Upvotes: 17

Views: 15635

Answers (4)

Ivan Gonzalez
Ivan Gonzalez

Reputation: 576

Delete all your module imports and also your tests import that also import your modules:

import sys

for key in list(sys.modules.keys()):
    if key.startswith("your_package_name") or key.startswith("test"):
        del sys.modules[key]

you can use this as a fixture by configuring on your conftest.py file a fixture using the @pytest.fixture decorator.

Upvotes: 6

Jeremy Barnes
Jeremy Barnes

Reputation: 662

To reload a module, try using the reload() from library importlib

Example:

from importlib import reload
import some_lib
#do something
reload(some_lib)

Also, launching each test in a new process is viable, but multiprocessed code is kind of painful to debug.

Example

import some_test
from multiprocessing import Manager, Process

#create new return value holder, in this case a list
manager = Manager()
return_value = manager.list()
#create new process
process =  Process(target=some_test.some_function, args=(arg, return_value))
#execute process
process.start()
#finish and return process
process.join()
#you can now use your return value as if it were a normal list, 
#as long as it was assigned in your subprocess

Upvotes: 6

Sam Daniel
Sam Daniel

Reputation: 1902

Once I had similar problem, quite bad design though..

@pytest.fixture()
def module_type1():
    mod = importlib.import_module('example')
    mod._init(10)
    yield mod
    del sys.modules['example']

@pytest.fixture()
def module_type2():
    mod = importlib.import_module('example')
    mod._init(20)
    yield mod
    del sys.modules['example']


def test1(module_type1)
    pass

def test2(module_type2)
    pass

The example/init.py had something like this

def _init(val):
    if 'sample' in globals():
        logger.info(f'example already imported, val{sample}' )
    else:
        globals()['sample'] = val
        logger.info(f'importing example with val : {val}')

output:

importing example with val : 10
importing example with val : 20

No clue as to how complex your package is, but if its just global variables, then this probably helps.

Upvotes: 2

ahll
ahll

Reputation: 2417

I have the same problem, and found three solutions:

  1. reload(some_lib)
  2. patch SUT, as the imported method is a key and value in SUT, you can patch the SUT. Example, if you use f2 of m2 in m1, you can patch m1.f2 instead of m2.f2
  3. import module, and use module.function.

Upvotes: 0

Related Questions