CODEWITHSUNDEEP
pythoniterator
Mkoch
Mkoch

Reputation: 2024

Python iterator when underlying object changes

I would like to know what's the general behaviour of an iterator if the underlaying object changes during the iteration.

Using a simple mutable list, it seems obvious: the iterator will try to follow on the next element if any, and send StopIteration if the end is reached.

>>> l = range(10)
>>> a = iter(l)
>>> a.next()
0
>>> a.next()
1
>>> a.next()
2
>>> l[3]='a'
>>> a.next()
'a'
>>> a.next()
4
>>> del l[5]
>>> a.next()
6
>>> a.next()
7
>>> a.next()
8
>>> a.next()
9
>>> a.next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

This is self-explanatory so far. What I don't understand is, that if I append a new element, the iterator will still return StopIteration.

>>> l.append(11)
>>> a.next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

If I do the same before reaching the end:

>>> l=[1]
>>> a=iter(l)
>>> a.next()
1
>>> l.append(2)
>>> a.next()
2

How is this working under the hood, and what is the expected behaviour of a more complex mutable iterable object? (e.g. think of an object representing a graph, which then can be iterated over, using a traversal algorithm. What is supposed to happen then if nodes are added/removed while iterating?)

Upvotes: 3

Views: 831

Answers (3)

Ethan Furman
Ethan Furman

Reputation: 69100

There are some iterators that can yield more data even after a StopIteration has been raised; such iterators are considered broken.

This does not mean the iterator is at fault -- it means that if you use such an iterator without caution you can end up with bugs and broken code.

Upvotes: 0

Ethan Furman
Ethan Furman

Reputation: 69100

There are three typical behaviours of changing an object while iterating over it:

  • the new data will be returned
  • the new data will be ignored
  • old data will be skipped

In other words: the actual behaviour is undefined.

Changing objects while iterating over them was such a common problem that in Python 3 the types set and dict (and possibly others) were changed to immediately raise an error if additions or removals were detected during iteration.

Upvotes: 0

Phillip
Phillip

Reputation: 13668

There's a comment on that particular issue in PEP 234 (Iterators):

Once a particular iterator object has raised StopIteration, will it also raise StopIteration on all subsequent next() calls?

Some say that it would be useful to require this, others say that it is useful to leave this open to individual iterators. Note that this may require an additional state bit for some iterator implementations (e.g. function-wrapping iterators).

Resolution: once StopIteration is raised, calling it.next() continues to raise StopIteration.

Note: this was in fact not implemented in Python 2.2; there are many cases where an iterator's next() method can raise StopIteration on one call but not on the next. This has been remedied in Python 2.3.

Upvotes: 2

Related Questions