How to use sys.meta_path with Python 3.x?

Question

given is the one of the code snippet/ sample which we are using in my old application, I have to migrate this code from python2.7 to python3.7. If we run on python3 getting error as "KeyError: 'my_virtual_module'" but on python2 its giving an o/p "hello world". How can I migrate this code to compatibtle with python2.7 and python3.x. I could notice find_module is executing in python2.7 but not in python3.x, what is alternative method in python3.x. Thanks in advance. Can you please help to use append'sys.meta_path, which is an extremely valuable tool that can be used to implement import hooks.

I have tried to run this code in python3

class VirtualModule(object):
   def hello(self):
      return 'Hello World!'   
class CustomImporter(object):

   virtual_name = 'my_virtual_module'

   def find_module(self, fullname, path=None):

      if fullname ==  self.virtual_name:
         return self
      return None

   def load_module(self, fullname):
      if fullname != self.virtual_name:
         raise ImportError(fullname)

      return VirtualModule()

if __name__ == '__main__':
   # Add our import hook to sys.meta_path
   sys.meta_path.append(CustomImporter())

   # Let's use our import hook
   import my_virtual_module
   print my_virtual_module.hello()

  File "Main.py", line 49, in <module>
    import my_virtual_module
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 628, in _load_backward_compatible
KeyError: 'my_virtual_module'

Answer 1

This article has a pretty good description:

https://dev.to/dangerontheranger/dependency-injection-with-import-hooks-in-python-3-5hap

In short, you create a meta path finder class, which returns a module spec, which contains a loader:

import importlib.abc
import importlib.machinery

class DependencyInjectorFinder(importlib.abc.MetaPathFinder):
    def __init__(self, loader):
        # we'll write the loader in a minute, hang tight
        self._loader = loader
    def find_spec(self, fullname, path, target=None):
        """Attempt to locate the requested module
        fullname is the fully-qualified name of the module,
        path is set to __path__ for sub-modules/packages, or None otherwise.
        target can be a module object, but is unused in this example.
        """
        if self._loader.provides(fullname):
            return self._gen_spec(fullname)
    def _gen_spec(self, fullname):
        spec = importlib.machinery.ModuleSpec(fullname, self._loader)
        return spec

Then, define a loader, which has procedures, create_module and exec_module that specify how to create the module object:

class DependencyInjectorLoader(importlib.abc.Loader):
    _COMMON_PREFIX = "myapp.virtual."
    def __init__(self):
        self._services = {}
        # create a dummy module to return when Python attempts to import
        # myapp and myapp.virtual, the :-1 removes the last "." for
        # aesthetic reasons :) 
        self._dummy_module = types.ModuleType(self._COMMON_PREFIX[:-1])
        # set __path__ so Python believes our dummy module is a package
        # this is important, since otherwise Python will believe our
        # dummy module can have no submodules
        self._dummy_module.__path__ = []
    def provide(self, service_name, module):
        """Register a service as provided via the given module
        A service is any Python object in this context - an imported module,
        a class, etc."""
        self._services[service_name] = module
    def provides(self, fullname):
        if self._truncate_name(fullname) in self._services:
            return True
        else:
            # this checks if we should return the dummy module,
            # since this evaluates to True when importing myapp and
            # myapp.virtual
            return self._COMMON_PREFIX.startswith(fullname)
    def create_module(self, spec):
        """Create the given module from the supplied module spec
        Under the hood, this module returns a service or a dummy module,
        depending on whether Python is still importing one of the names listed
        in _COMMON_PREFIX.
        """
        service_name = self._truncate_name(spec.name)
        if service_name not in self._services:
            # return our dummy module since at this point we're loading
            # *something* along the lines of "myapp.virtual" that's not
            # a service
            return self._dummy_module
        module = self._services[service_name]
        return module
    def exec_module(self, module):
        """Execute the given module in its own namespace
        This method is required to be present by importlib.abc.Loader,
        but since we know our module object is already fully-formed,
        this method merely no-ops.
        """
        pass
    def _truncate_name(self, fullname):
        """Strip off _COMMON_PREFIX from the given module name
        Convenience method when checking if a service is provided.
        """
        return fullname[len(self._COMMON_PREFIX):]

This example defines a few additional procedures that operate for how the specific finder/loader handles the module creation.