CODEWITHSUNDEEP
pythonpython-importpython-module
demonguy
demonguy

Reputation: 2017

How to import two versions of the same python module at the same time?

Suppose I have two versions of a python package, say "lib". One is in folder ~/version1/lib and the other is in ~/version2/lib. I'm trying to load both packages in one session by doing this:

sys.path.insert(0, '~/version1')
import lib as a

sys.path.insert(0, '~/version2')
import lib as b

But it doesn't work, because of cache, b will be the same as a.

Is there anyway to do it? Maybe use hook in sys.meta_path? I didn't figure it out.

Or is there anyway to delete cache of imported modules?

Upvotes: 13

Views: 6047

Answers (3)

MegaIng
MegaIng

Reputation: 7886

Using the new importlib interfaces, this is possible relatively simply:

import sys
from importlib.util import spec_from_file_location, module_from_spec
from pathlib import Path


def secondary_import(path: Path, name: str = None):
    if name is None:
        name = path.with_suffix('').name

    old_modules = {n: m for n, m in sys.modules.items() if n == name or n.startswith(name + ".")}
    for n in old_modules:
        del sys.modules[n]

    spec =  spec_from_file_location(name, str(path))
    module = module_from_spec(spec)
    sys.modules[name] = module
    spec.loader.exec_module(module)
    our_modules = {n: sys.modules[n] for n in old_modules}
    sys.modules.update(old_modules)
    return module, our_modules

This can then be used like this:


sys.path.insert(0, '~/version1')
import lib as a


sys.path.pop(0)

b, _ = secondary_import(os.path.join('~/version2', os.path.relpath(a.__path__, '~/version1')))

Note that this can easily break, especially if lib does anything weird with imports. This includes:

  • Modifying truly global state (e.g. sys.path)
  • Modifying global state of other modules
  • Local relative imports inside of functions (like are sometimes used to fix circular dependencies)
  • Dynamic imports
  • Uses something from importlib.resources or importlib.metadata
  • Non-basic-python files like C extensions or namespace packages

A few of these can be worked around, a few can't. I might create a package to see what can be done in a general way.

Upvotes: 0

Barun Sharma
Barun Sharma

Reputation: 1468

Upvotes: -3

Klaus D.
Klaus D.

Reputation: 14369

You have to import it from one level higher:

import version1.my_lib as a
import version2.my_lib as b

Also make sure that in all the folder there is a __init__.py.

Upvotes: 5

Related Questions