CODEWITHSUNDEEP
pythonpython-2.7
MariusSiuram
MariusSiuram

Reputation: 3644

Binding functions (`exec`uted at runtime) to a module (created at runtime)

I am developing an application that uses a "runtime generated module". The fashion is the following:

import imp
import sys

a_module = imp.new_module("runtime_module")
sys.modules["runtime_module"] = a_module

For a simplified two-function example, I have them stored in two different txt files. I should read from file, but to make it more simple:

# First function
source_func1 = """def func1_implementation():
    func2()
"""

# Second function
source_func2 = """def func2_implementation():
    pass
"""

And, to "populate" the class, I load the code into source_func1 and source_func2` and do the following:

exec compile(source_func1, "__remote__func1__", "exec")
exec compile(source_func2, "__remote__func2__", "exec")
a_module.func1 = func1_implementation
a_module.func2 = func2_implementation

I would like to be able to do the following:

import runtime_module
runtime_module.func1()

However...

----------------------
NameError  Traceback (most recent call last)
<ipython-input-11-4a304d14319a> in <module>()
      1 import runtime_module
----> 2 runtime_module.func1()

/.../__remote__func1__ in func1_implementation()

NameError: global name 'func2' is not defined

How can I manage the namespace? Is there a pythonic way to achieve what I am trying to do? I don't have a lot of freedom on the txt files (from which I execute the source file). I also don't fully understand "where" are the namespaces/locals/globals, so maybe I am overlooking some easy solution.

Upvotes: 1

Views: 38

Answers (1)

abarnert
abarnert

Reputation: 365777

Your main problem is that when you execute a function definition, it binds the global environment at that time to its __globals__ attribute, and when the function is later called, that's what's used as the frame's globals.

So, you can look at func1_implementation.__globals__, and it's obviously your module's globals, not a_module's globals.

You can solve this just by doing:

exec compile(source_func1, "__remote__func1__", "exec") in a_module.__dict__
exec compile(source_func2, "__remote__func2__", "exec") in a_module.__dict__

a_module.func1 = a_module.func1_implementation
a_module.func2 = a_module.func2_implementation

Those last two lines replace the setattr(…) bit, which is just obfuscating the same thing. a_module.func1 = <something> is the same thing as setattr(a_module, "func1", <something>), and locals()["func1_implementation"] is the same thing as func1_implementation.

Notice that this puts func1_implementation into a_module. Which isn't as bad as putting it into your own top-level module, but it's still not great.

You can solve that in a number of ways: just extract the code objects and use types.FunctionType to construct new function objects bound to the right namespace, use a new namespace and cram it into a_module after the fact, or, probably simplest, just this:

del a_module.func1_implementation
del a_module.func2_implementation

This is, after all, the same thing you do in normal modules when you need a helper function to build a class or something but don't want to keep it around in the module after it's built…

Upvotes: 1

Related Questions