CODEWITHSUNDEEP
pythonpython-3.xpython-importlibdynamic-import
Josh
Josh

Reputation: 1429

Python: Dynamically import module's code from string with importlib

I wish to dynamically import a module in Python (3.7), whereby the module's code is defined within a string.

Below is a working example that uses the imp module, which is deprecated in favour of importlib (as of version 3.4):

import imp

def import_code(code, name):
    # create blank module
    module = imp.new_module(name)
    # populate the module with code
    exec(code, module.__dict__)
    return module

code = """
def testFunc():
    print('spam!')
"""

m = import_code(code, 'test')
m.testFunc()

Python's documentation states that importlib.util.module_from_spec() should be used instead of imp.new_module(). However, there doesn't seem to be a way to create a blank module object using the importlib module, like I could with imp.

How can I use importlib instead of imp to achieve the same result?

Upvotes: 10

Views: 4554

Answers (3)

boludoz
boludoz

Reputation: 68

import importlib.util
import urllib.request
import sys
from cryptography.fernet import Fernet

def import_code(url, import_as=None, key_string=None):
    
    # set import_as to the name of the url if it is None
    if import_as == None:
        import_as = url.split('/')[-1].split('.')[0]

    # throw error if import_as is not a string
    if not isinstance(import_as, str):
        raise TypeError('import_as must be a string')
    
    # throw error if url is not a string
    if not isinstance(url, str):
        raise TypeError('url must be a string')
    
    # throw error if url is empty
    if not url:
        raise ValueError('url must not be empty')
    
    try:
        # Download the module code from the URL
        response = urllib.request.urlopen(url)
        module_code = response.read().decode('utf-8')

        # Decrypt the module code if it is encrypted with Fernet and try except
        if key_string!=None:
            fernet_obj = Fernet(key_string)
            try:
                module_code = fernet_obj.decrypt(module_code)
            except:
                pass
        
        # Execute the module code in the current scope
        if import_as == '':
            exec(module_code, globals())
            return None

        # Create a new module object
        spec = importlib.util.spec_from_loader(import_as, loader=None)
        module = importlib.util.module_from_spec(spec)
        exec(module_code, module.__dict__)
        sys.modules[import_as] = module
        exec(f'{import_as} = module')
        return module
    except:
        raise

In that example I show you how to download a file from the internet, which may be encrypted with Fernet and I teach you how to execute it. I used it for a project, you can modify it and have it directly import a text string.

Ways to use:

# Take from url and file name
import_code(url, import_as=None)
import hello_word
hello_word.hello_word()

# Take custom name
hello_word = import_code(url)
hello_word.hello_word()

# Global import
import_code(url, import_as='')
hello_word()

Upvotes: 0

LVitya
LVitya

Reputation: 567

According to Python documentation module_from_spec()

importlib.util.module_from_spec(spec)

...

This function is preferred over using types.ModuleType to create a new module as spec is used to set as many import-controlled attributes on the module as possible.

Here is what I came up with to load the module from source code located in github repo. It is a way without writing the file to disk.

import requests
url = "https://github.com/udacity/deep-learning-v2-pytorch/raw/master/intro-to-pytorch/helper.py"
r = requests.get(url)

import importlib.util
spec = importlib.util.spec_from_loader('helper', loader=None, origin=url)
helper = importlib.util.module_from_spec(spec)
exec(r.content, helper.__dict__)

helper.view_classify() # executes function from github file

Upvotes: 7

Right leg
Right leg

Reputation: 16730

You can simply instantiate types.Module:

import types
mod = types.ModuleType("mod")

Then you can populate it with exec just like you did:

exec(code, mod.__dict__)
mod.testFunc() # will print 'spam!'

So your code will look like this:

import types

def import_code(code, name):
    # create blank module
    module = types.ModuleType(name)
    # populate the module with code
    exec(code, module.__dict__)
    return module

code = """
def testFunc():
    print('spam!')
"""

m = import_code(code, 'test')
m.testFunc()

As commented by @Error - Syntactical Remorse, you should keep in mind that exec basically executes whatever code is contained in the string you give it, so you should use it with extra care. At least check what you're given, but it'd be good to use exclusively predefined strings.

Upvotes: 10

Related Questions