Python exec() function broken in versions above 2.7? Error: 'name' not defined

Question

I've discovered a strange behaviour of python exec() function. Here is the code:

variables = {
    ('foo', 6),
    ('bar', 42)
}

def func():
    for varData in variables:
        varName, varValue = varData
        localVarToEvaluate = varName + ' = varValue'
        try:
            #exec(localVarToEvaluate, globals(), locals())
            exec(localVarToEvaluate)
        except Exception as err:
            print(str(err))

        if varName not in locals():
            print("Variable names '", varName, "can't be found in local scope!")

    if 'foo' in locals():
        print("'foo' OK:", foo)  # exception here
    else:
        print("'foo' not available!")

    if 'bar' in locals():
        print("'bar' OK:", bar)
    else:
        print("'bar' not available!")

func()

I would expect variables foo and bar to be created and printed at the end by exec() call, which is the case with Python 2.7. Everything above (tested on 3.3, 3.4, 3.6, and 3.7) throws exception, that foo is not defined:

Exception has occurred: NameError
    name 'foo' is not defined

Strange thing here is that foo and bar is seen by running locals(), globals() or dir() (also confirmed by if statements), however, it is not seen by the code/interpreter. Even stranger, debugging this script and resolving any variable is successfull (I've set a breakpoint on # exception here and type foo in Debug window using VS Code. foo is correctly resolved with a value of '6'.

If the same code (stuff inside function func()) is not wrapped in function, this works as expected, foo and bar are printed out.

Any idea what is happening here?

---

UPDATE: I've further simplify this problem:

# take 1, create local variable 'foo' with value 6. Not in function.
varName = 'foo'
varValue = 42
localVarToEvaluate = varName + ' = varValue'

try:
    exec(localVarToEvaluate)
except Exception as err:
    print(str(err))

if 'foo' in locals():
    # print(locals()['foo']) # (1)
    # print(foo)  # (2)
    print("'foo' OK:", foo)  # (3)


# take 2, create local variable 'bar' with value 42
def func2():
    varName = 'bar'
    varValue = 42
    localVarToEvaluate = varName + ' = varValue'

    try:
        exec(localVarToEvaluate)
    except Exception as err:
        print(str(err))

    if 'bar' in locals():
        # print(locals()['bar']) # (1)
        # print(bar)  # (2)
        #print("'bar' OK:", bar)  # (3)
        pass # uncomment any line above

func2()

When this code executes, first:

'foo' OK: 6

is printed, than this exception is raised:

Exception has occurred: NameError
name 'bar' is not defined
  ...

Note that both codes are identical, except that 'bar' variable is created inside function func2().

What I am interested in are not workarounds but explanation, why is this so and why points (1) works, while (2) and (3) does not. Note that bar variable is seen in locals(), while it is not accessible by directly calling it - but only if it is created inside function!

Answer 1

I've reported a bug on a Python Issue Tracker, and the official answer is:

This is currently by design, which means 3.8 is likely the only viable place it can change. It's also not Windows specific so I removed that component (people may remove themselves from nosy).
...
Currently it's basically a read-only proxy, as locals are optimized within functions which is why you can't see updates via the duct.

Bottom line, exec() used in this way is useless inside functions.

Answer 2

Lots of unclear issues about this question solved with OP. See answer edits. It boils down to (importing) hooking locals (variables, defs, classes) such that they are available for use inside a definition.

See answer below with inline comments what is what and why.

# take 1, create local variable 'foo' with value 6. Not in function.

# >>> code is executed in local-scope <<<

varName = 'foo'
varValue = 42
localVarToEvaluate = varName + ' = varValue'

try:
    exec(localVarToEvaluate)    # (0) dict item becomes {varName : varValue}
    print (localVarToEvaluate)  # (1)  prints > foo = varValue < dict item
except Exception as err:
    print(str(err))

if 'foo' in locals():
    print(locals()['foo'])      # (2)  prints > 42             <             value
    print(foo)                  # (3)  prints > 42             <             value
    print("'foo' OK:", foo)     # (4)  prints > 'foo' OK: 42   < stringtext, value


# take 2, create local variable 'bar' with value 42

def func2(self):

    # >>> code executed inside function and not local-scope <<<

    varName = 'bar'
    varValue = 42
    localVar2Evaluate = varName + ' = varValue'

    try:
        exec(localVar2Evaluate)    # (5) dict item becomes {varName : varValue}
        print (localVar2Evaluate)  # (6) prints > bar = varValue < dict item
    except Exception as err:
        print(str(err))

    print ('local-scope :', '\n', locals())  # (7) {'bar': 42, 'localVar2Evaluate': 'bar = varValue', 'varValue': 42, 'varName': 'bar'}

    if 'bar' in locals():
        print(locals()['bar'])     # (1)
        print(bar)                 # (2)  < --- python is not looking here in the locals() but inside the def for variable `bar` which is not made unless you give it access (hook or reference) via e.g. self.
        print("'bar' OK:", bar)    # (3)
       # pass # uncomment any line above

x = 'a scotch... lets drink.. mystery solved!'
bar = "the local 'bar' variable is now available inside def func2()..  It is: %s" % x
func2(bar)

As you can see I (import) create a hook to a local variable with varName 'bar' to be used inside the definition using self. It can be any name t.b.h. See pydocs on self, etc.

The result:

bar = varValue
local-scope : 
 {'localVar2Evaluate': 'bar = varValue', 'varValue': 42, 'bar': 42, 'self': "the local 'bar' variable is now available inside def func2()..  It is: a scotch... lets drink.. mystery solved!", 'varName': 'bar'}
42
the local 'bar' variable is now available inside def func2()..  It is: a scotch... lets drink.. mystery solved!
'bar' OK: the local 'bar' variable is now available inside def func2()..  It is: a scotch... lets drink.. mystery solved!

If print('\n\n', locals()) below func() you get the following printresult:

1. 'bar': "the local 'bar' variable is now available inside def func2().. It is: a scotch... lets drink.. mistery solved!"
2. 'localVarToEvaluate': 'foo = varValue'
3. 'varValue': 42
4. 'foo': 42
5. 'varName': 'foo'
6. 'x': 'a scotch... lets drink.. mistery solved!'
7. 'func2': "<"function func2 at 0x000002B070027F28">" # without " outside ">".

At bullet 7 you see func2 linked.

UPDATE 4:

Switching between python 2.7.16 and 3.5.2 revealed no change for the locals() dict and ONE change in the globals() dict as shown below.

In 2.7: 'variables': set([('bar', 42), ('foo', 6)])

In 3.5: 'variables': {('bar', 42), ('foo', 6)}

... this set() looks to me the reason why it no longer works what you addressed in 3.5.

I did test it with adapting your script:

import sys

print (sys.version)

variables = {('foo', 6), ('bar', 42)}

def func():
    for varData in variables:
        varName, varValue = varData
        localVarToEvaluate = varName + ' = varValue'
        try:
            exec(localVarToEvaluate)
            print ('t2\n', locals())
        except Exception as err:
            print(str(err))

        if varName not in globals():
            print("Variable names '", varName, "can't be found in global scope!")

    if 'foo' in globals():
        print("'foo' OK:", foo)  # exception here
    else:
        print("'foo' not available!")

    if 'bar' in globals():
        print("'bar' OK:", bar)
    else:
        print("'bar' not available!")

print ('t1\n', globals())

func()

Then ... it could still be exec. So I disabled running func() and the difference in globals() remained. SO I think its a difference in globals() function, rather exec.