CODEWITHSUNDEEP
pythonpython-importlib
FamousSnake
FamousSnake

Reputation: 451

Why undeclared variables are present in `globals()` after a reload and is it safe to use them to identify a reload?

I've found that the snippet below when reloading a module test unexpectedly has all variables already defined in globals()/locals().

Why this happens?

I've noticed this "xxx" in locals() pattern a lot in Blender Python scripts as typically people use it to check whether module was reloaded before (you typically reload those scripts a lot during the development). Since everyone's using it, I guess it should work most of the times.

But is it really safe to use this caveat to identify whether module was already loaded before (any ideas of cases when it wouldn't work?)? I mean not just for the development and testing stuff as it does looks like more of an implementation detail that could break in the production.

# prints False, False, False
import test
import importlib

# prints True, True, True
importlib.reload(test)

test.py:

print("a" in globals(), "b" in globals(), "c" in globals())

# NameError: name 'a' is not defined
# print(a)

a = 25
b = 35
c = 45

Upvotes: 0

Views: 59

Answers (2)

tdelaney
tdelaney

Reputation: 77407

When you load a module, its top level code is executed and, besides variables created by Python itself when loading the module, all variables are bound during that execution. In your example, a doesn't exist until a = 25 is executed.

Now, from the importlib.reload doc we read

When a module is reloaded, its dictionary (containing the module’s global variables) is retained.

Further

the module-level code re-executed, defining a new set of objects which are bound to names in the module’s dictionary

This means that module level variables are not deleted but they may be overwritten when the new module executes. Your test.py module does the right thing. It tests for the existence of a variable before it has been set in the executing code. If a is set before that first assignment, it's likely that the module has been reloaded, because how else could it get there?

But that's not a total guarantee. Something could have loaded the module with importlib.__import__ and supplied its own globals dict. Or perhaps a later revision of the code starts using that variable differently. Imagine a function is updated to set that variable without realizing it injects a bug. Or another that deletes that variable.

I think it's a reasonable technique in production code when used carefully. For instance, use some horribly named variable with a canned value that is clearly used only for that one purpose. But recognize there is some fragility.

Fiddling around a bit, I like

# Used only to get the number of reloads for this module.
try:
    _reload_counter += 1
except NameError:
    _reload_counter = 1

Upvotes: 1

jsbueno
jsbueno

Reputation: 110591

As stated by @jasonharper in the comments: yes, the __dict__ of the module is retained when importlib.reload is used. (From the importlib.reload() documentation: "The names in the module namespace are updated to point to any new or changed objects" and "When a module is reloaded, its dictionary (containing the module’s global variables) is retained". Quoted from the docs by @jasonharper)

For regular code that is never an issue, since all names that a module will use have to be defined first - either as an assignemtn of as a function or class declaration. This assignment will update the dict and replace the previous references that is there.

But checking if names as strings are present in globals() like you are doing will detect the pre-existing names, and can be used to know if the module is in the middle of a reloading operation.

However, if the module is maually "deleted" from sys.modules prior to reloading, then it will run with a fresh globals() dictionary. This can only be done intentionally - so if you plan to base any behavior on whether an item is present in globals() it is ok.

To see the behavior I mean, try this snippet:

# prints False, False, False
import test
import importlib

# prints True, True, True
importlib.reload(test)

import sys
del sys.modules("test")
importlib.reload(test)

# or even:
del sys.modules("test")
# module is reloaded, even without a `.reload()` call,
# as it is not in sys.modules anymore
import test

Upvotes: 1

Related Questions