CODEWITHSUNDEEP
pythongeneratorpython-internals
overexchange
overexchange

Reputation: 1

How does a generator function work internally?

Below is a generator function.

def f():
   x=1
   while 1:
      y = yield x
      x += y

Does this generator function (f) get implemented, internally, as shown below?

class f(collections.Iterable):
   def __init__(self):
      self.x = 1
   def __iter__(self):
      return iter(self)
   def __next__(self):
      return self.x
   def send(self, y):
      self.x += y
      return self.next()

Edit:

This is the answer for my question.

Upvotes: 10

Views: 3701

Answers (1)

Raymond Hettinger
Raymond Hettinger

Reputation: 226754

Internally, a generator works about the same as a regular function call. Under-the-hood, running generators and running functions use mostly the same machinery.

When you call either a function or a generator, a stackframe is created. It has the local variables (including the arguments passed into the function), a code pointer to the active opcode, and a stack for pending try-blocks, with-blocks, or loops.

In a regular function, execution begins immediately. When return is encountered, the final result is kept and the stackframe is freed along with everything it referenced.

In a generator function, the stackframe is wrapped in a generator-iterator object and returned immediately. The code in the generator function only runs when called by next(g) or g.send(v). Execution is suspended when yield is encountered.

One way to think of generators is that they are like functions that can be paused with yield and resumed with g.next(). The stackframe is kept alive, so resuming a running generator is much cheaper than making a new function call which has to build a new frame on every invocation.

Upvotes: 33

Related Questions