Tricky Python question about variable scope

Question

I often have variables in my Python projects that I need to share between modules. I know this can be done with args to function calls, but sometimes, it's more convenient to create a single global_vars.py module, and add any variables that need to be shared there, which is what I often end up doing. These variables can then be imported from any other module, and easily shared. This generally works fine for me. Sometimes though, some unexpected stuff happens.

Ex:

I have 3 files:

main.py:

from global_vars import var
from mod import modify_variable

print(f"{hex(id(var))} - address of var in main module after import")
modify_variable()
print(f"{hex(id(var))} - address of var in main module after modify_variable() call")

global_vars.py:

var = 'hi'
print(f"{hex(id(var))} - address of var during import")

mod.py: from global_vars import var

def modify_variable():
    global var

    print(f"{hex(id(var))} - address of var before modifying it in modify_variable()")
    var = 'hello'
    print(f"{hex(id(var))} - address of var after modifying it in modify_variable()")

If you run main.py with Python3, you get output that looks like this:

0x7f0f993bb7f0 - address of var during import
0x7f0f993bb7f0 - address of var in main module after import
0x7f0f993bb7f0 - address of var before modifying it in modify_variable()
0x7f0f993bb870 - address of var after modifying it in modify_variable()
0x7f0f993bb7f0 - address of var in main module after modify_variable() call

Basically, everything behaves as expected until we call modify_variable. Within modify_variable, the address of var starts out as we expect. Then we assign a new string to it. This does a few things:

• It creates a brand new string in memory
• It changes var to point to the address of this new string
• The garbage collector would presumably, at some point, clean up the old string's address

Given this, I would expect that the last check of the address of var would also point to the address of the new string. Therefore, I would expect the result to look like this:

0x7f0f993bb7f0 - address of var during import
0x7f0f993bb7f0 - address of var in main module after import
0x7f0f993bb7f0 - address of var before modifying it in modify_variable()
0x7f0f993bb870 - address of var after modifying it in modify_variable()
0x7f0f993bb870 - address of var in main module after modify_variable() call

but it doesn't. var within modify_variable, and var within main.py now point to completely different addresses, and can no longer share the same data.

What happened? From what I've read, if I hadn't used global var in mod.py, it was possible that I would have created a local variable with the same name as the global var variable, which could cause the symptoms above, but using global var as I do ensures I'm dealing with global_vars.var, doesn't it?

Answer 1

The spec

Python doesn't really have global variables. Here's what python defines as global:

If a name is bound at the module level, it is a global variable. (The variables of the module code block are local and global.)

And the global statement means:

It means that the listed identifiers are to be interpreted as globals.

i.e. as module-level variables.

If you look at what import does:

1. find the module specified in the from clause, loading and initializing it if necessary;

2. for each of the identifiers specified in the import clauses:

   [...]

   4. [...] a reference to that value is stored in the local namespace, using the name in the as clause if it is present, otherwise using the attribute name

Your case

So, global_vars.var points to address 0x7f0f993bb7f0, and when you import it to mod, it becomes mod.var which also points to 0x7f0f993bb7f0.

When you do global var you tell the Python parser to bind var to mod.var, and then with var = 'hello', you make mod.var point to 0x7f0f993bb870.

But in main.py, var binds to main.var, which was assigned by the import statement to global_vars.var, namely 0x7f0f993bb7f0.

What to do

Implement globals as attributes of a single global object:

g.py:

class Global:
    pass

g = Global()

setattr(g, 'var', 'hi')

print(f"{hex(id(g.var))} - address of var during import")

mod.py:

from g import g

def modify_variable():
    print(f"{hex(id(g.var))} - address of var before modifying it in modify_variable()")
    g.var = 'hello'
    print(f"{hex(id(g.var))} - address of var after modifying it in modify_variable()")

main.py:

#!/usr/bin/python3.6
from g import g
from mod import modify_variable

print(f"{hex(id(g.var))} - address of var in main module after import")
modify_variable()
print(f"{hex(id(g.var))} - address of var in main module after modify_variable() call")

Output:

0x7ff2a47a1998 - address of var during import
0x7ff2a47a1998 - address of var in main module after import
0x7ff2a47a1998 - address of var before modifying it in modify_variable()
0x7ff2a47ae500 - address of var after modifying it in modify_variable()
0x7ff2a47ae500 - address of var in main module after modify_variable() call