CODEWITHSUNDEEP
pythoninheritanceoverridingiterable
Muhammed Hasan Celik
Muhammed Hasan Celik

Reputation: 682

Why does overriding __iter__ throws RecursionError?

I want to override the __iter__ method of the ExternalIter which is defined in an external library so I cannot change the source code of ExternalIter class. I implemented two different ways of overriding in the MyIter2 and MyIter3.

class ExternalIter:
    
    def __init__(self):
        self.x = iter([
            (1, 2),
            (3, 4),
            (5, 6),
            (7, 8),
            (9, 10)
        ])
        
    def __iter__(self):
        return self
        
    def __next__(self):
        return next(self.x)
    
class MyIter2(ExternalIter):
    
    def __iter__(self):
        while True:
            try:
                i, j = super().__next__()
                yield i
                yield j
            except StopIteration:
                break
    
class MyIter3(ExternalIter):
    
    def __iter__(self):
        for i, j in super().__iter__():
            yield i
            yield j

Original behavior of the ExternalIter is the following:

for i in ExternalIter():
    print(i)
# (1, 2)
# (3, 4)
# (5, 6)
# (7, 8)
# (9, 10)

I want to override the behavior of the __iter__ as in the following example:

for i in MyIter2():
    print(i)
# 1
# 2
# 3
# 4
# 5
# 6
# 7
# 8
# 9
# 10

MyIter2 works as excepted. However, an alternative way of overriding as I did in MyIter3 throws RecursionError.

for i in MyIter3():
    print(i)
---------------------------------------------------------------------------
RecursionError                            Traceback (most recent call last)
<ipython-input-8-4e1114201cac> in <module>
----> 1 for i in MyIter3():
      2     print(i)

<ipython-input-5-82da3324a8a3> in __iter__(self)
     30 
     31     def __iter__(self):
---> 32         for i, j in super().__iter__():
     33             yield i
     34             yield j

... last 1 frames repeated, from the frame below ...

<ipython-input-5-82da3324a8a3> in __iter__(self)
     30 
     31     def __iter__(self):
---> 32         for i, j in super().__iter__():
     33             yield i
     34             yield j

RecursionError: maximum recursion depth exceeded while calling a Python object

Can you explain why the second implementation of overriding throws error while the first way works? How can I override __iter__ with using super().__iter__() without getting RecursionError?

Can you explain how __iter__ works in this context of inheritance to understand why super().__iter__() refers to the self, not the parent?

Upvotes: 3

Views: 213

Answers (1)

chepner
chepner

Reputation: 531335

super().__iter__() returns self before the for loop makes its own implicit call of iter(self), which kicks off the infinite recursion.

Because of how a for loop works, I don't seen any way to use one here. You can use an explicit while loop instead to avoid calls to __iter__.

def __iter__(self):

    while True:
        try:
            i, j = next(self)
        except StopIteration:
            break
        yield i
        yield j

This doesn't treat self as an iterable, only as an iterator.

Upvotes: 3

Related Questions