CODEWITHSUNDEEP
pythonpython-3.xclosures
Malik A. Rumi
Malik A. Rumi

Reputation: 2025

When does a closure in Python finish execution or go out of scope?

I am trying to wrap my brain around closures. I’ve been looking at a lot of sources, but this question is based mainly on the explanation in Wikipedia.

I think I get the idea of a callback as a closure, to handle events that happen after something in the outer function has happened / been called. I guess that’s what happening with deferreds in Scrapy. I’ve also seen examples in Python using the logging module.

Ok, but, but, you can’t call a closure on its own, right? Because that would be calling it outside the scope of where it was defined. Putting it another way, you can’t call a closure directly, you have to call the enclosing function which returns it. But if that’s the case, then the use case for closures is limited by its dependency on the outer function. The fact that it remembers the free variables of the enclosing environment after the outer function has ceased execution does not make the closure a little portable unit you can take and use anywhere and it will work without the outer function, right?

But if that’s right, then how can this also be right:

“...When the closure is entered at a later time, possibly with a different lexical environment, the function is executed with its non-local variables referring to the ones captured by the closure, not the current environment…” Wikipedia - same link - #(“Implementation and theory”)

This is why I am struggling so hard with closures. The idea that they remember values post execution sounds, at least to me, like it has a life independent of the enclosing function, as though you can call it or use it in “a different lexical environment”. But that’s not true if the enclosing function is right there with it, is it?

This example from datacamp helps spell out what I find confusing in a simpler example:

In Python, a function is also considered a object, which means that it can be returned and assigned to a variable. In the next example, you'll see that instead of inner() being called inside outer(), return inner is used. Then, outer() is called with a string argument and assigned to closure. Now, even though the functions inner() and outer() have finished executing, their message is still preserved. By calling closure(), the message can be printed.

def outer(message):
    # enclosing function
    def inner():
        # nested function
        print(message)
    return inner


closure = outer("Hello world!")
closure()
Hello world!

Notice that if you call closure without parentheses, only the type of the object will be returned. You can see that it's of the type function

__main__.outer.<locals>.inner.closure

<function __main__.outer.<locals>.inner>

Now, even though the functions inner() and outer() have finished executing, their message is still preserved.

In what sense have either inner() or outer() “finished executing” in this example? They haven’t even been called until you call closure()! What am I missing here? Thanks.

Upvotes: 0

Views: 722

Answers (2)

Conor Retra
Conor Retra

Reputation: 179

They haven’t even been called until you call closure()!

Actually, the statement closure = outer("Hello world!") calls the outer() function with the argument "Hello world!". At that time, the outer() function executes, thereby defining the inner() function and returning the function inner itself, assigning the function to the variable closure. At this point, outer() has "finished executing".

Next, by calling closure(), the function inner() is actually invoked, returning the preserved message.

Upvotes: 3

chepner
chepner

Reputation: 531225

The description is poorly worded; the function inner wasn't executed during the call to outer, just the def statement that defines it.

Among other attributes, the function object defined by def inner(): ... has one named _closure__ that contains the value to use when the nonlocal name message is looked up.

>>> closure.__closure__[0].cell_contents
'Hello'

To the best of my understanding (which is not, admittedly, great), each nonlocal variable in the definition of inner is numbered, and a tuple containing one cell object per nonlocal name is stored in __closure__. Each cell has an attribute cell_contents which contains the value held by the nonlocal name when outer returns.

Upvotes: 2

Related Questions