Can one mimic lexical scoping with context managers?

Question

In an earlier post I asked about ways to avoid the intermediate tmp variable in patterns like:

tmp = <some operation>
result = tmp[<boolean expression>]
del tmp

...where tmp is a pandas object. For example:

tmp = df.xs('A')['II'] - df.xs('B')['II']
result = tmp[tmp < 0]
del tmp

The bee in my bonnet about this pattern basically comes from a longing for honest-to-goodness lexical scoping¹ that just won't die, even after years of programming Python. In Python² I make do with explicit calls to del,

It occurs to me that it may be possible to use context managers to mimic lexical scoping in Python. It would look something like this:

with my(df.xs('A')['II'] - df.xs('B')['II']) as tmp:
    result = tmp[tmp < 0]

To be able to mimic lexical scoping, the context manager class would need to have a way to delete the variable in the calling scope that gets assigned the value returned by its (the context manager's) 'enter' method.

For example, with a generous dose of cheating:

import contextlib as cl

# herein lies the rub...
def deletelexical():
    try: del globals()['h']
    except: pass

@cl.contextmanager
def my(obj):
    try: yield obj
    finally: deletelexical()

with my(2+2) as h:
    print h
try:
    print h
except NameError, e:
    print '%s: %s' % (type(e).__name__, e)
# 4
# Name error: name 'h' is not defined

Of course, the problem is to implement deletelexical for real. Can it be done?

Edit: As abarnert pointed out, if there had been a pre-existing tmp in the surrounding scope, deletelexical would not restore it, so it could hardly be regarded as an emulation of lexical scoping. The correct implementation would have to save any existing tmp variables in the surrounding scope, and replace them at the end of the with-statement.

---

¹E.g., in Perl, I would have coded the above with something like:

my $result = do {
    my $tmp = $df->xs('A')['II'] - $df->xs('B')['II'];
    $tmp[$tmp < 0]
};

or in JavaScript:

var result = function () {
    var tmp = df.xs('A')['II'] - df.xs('B')['II'];
    return tmp[tmp < 0];
}();

Edit: In response to abarnert's post & comment: yes, in Python one could define

def tmpfn():
    tmp = df.xs('A')['II'] - df.xs('B')['II']
    return tmp[tmp < 0]

...and this would indeed prevent cluttering the namespace with the henceforth useless name tmp, but it does so by cluttering the namespace with the henceforth useless name tmpfn. JavaScript (and Perl also, BTW, among others) allows anonymous functions, while Python doesn't. In any case, I consider JavaScript's anonymous functions as a somewhat cumbersome way to get lexical scoping; it's certainly better than nothing, and I use it heavily, but it's nowhere near as nice as Perl's (and by the latter I mean not only Perl's do statement, but also the various other ways it provides to control scope, both lexical and dynamic).

²I don't need to be reminded of the fact that only an infinitesimally small fraction of Python programmers give a rat's tail about lexical scoping.

Answer 1

In your JavaScript equivalent, you do this:

var result = function () {
    var tmp = df.xs('A')['II'] - df.xs('B')['II'];
    return tmp[tmp < 0];
}();

In other words, in order to get an extra lexical scope, you're creating a new local function and using its scope. You can do the exact same thing in Python:

def tmpf():
    tmp = df.xs('A')['II'] - df.xs('B')['II']
    return tmp[tmp < 0]
result = tmpf()

And it has the exact same effect.

And that effect isn't what you seem to think it is. Going out of scope just means it's garbage that can be collected. That's exactly what a real lexical scope would give you, but it's not what you want (a way to deterministically destroy something at some point). Yes, it happens to usually do what you want in CPython 2.7, but that's not a language feature, it's an implementation detail.

But your idea adds some more problems on top of the problem with just using a function.

Your idea leaves everything defined or rebound within the with statement changed. The JS equivalent doesn't do that. What you're talking about is more like a C++ scope-guard macro than a let statement. (Some impure languages allow you to set!-bind new names within a let that will live on outside the let, and you could describe this as a lexical scope with an implicit nonlocal everything-but-the-let-names inside the body, but it's still pretty weird. Especially in a language that already has a strong distinction between rebinding and mutating.)

Also, if you already had a global with the same name tmp, this with statement would erase it. That's not what a let statement, or any other common form of lexical scoping, does. (And what if tmp were a local rather than global variable?)

If you want to simulate lexical scoping with a context manager, what you really need is a context manager that restores globals and/or locals on exit. Or maybe just a way to execute arbitrary code inside a temporary globals and/or locals. (I'm not sure if this is possible, but you get the idea—like getting the body of the with as a code object and passing it to exec.)

Or, if you want to allow rebinding to escape the scope, but not new bindings, walk the globals and/or locals and delete everything new.

Or, if you want to just delete a specific thing, just write a deleting context manager:

with deleting('tmp'):
    tmp = df.xs('A')['II'] - df.xs('B')['II']
    result = tmp[tmp < 0]

There's no reason to push the expression into the with statement and try to figure out what it gets bound to.