CODEWITHSUNDEEP
pythoniteratoriterable
oarfish
oarfish

Reputation: 4622

Why does calling list() on iterable change it?

Consider this code, wherein I use combinations and attempt to make a list out of them.

from itertools import combinations

t = (1,2,3,4)
print("t is %r" % (t,))
print("list(t) is %r" % list(t))
print("list(t) is %r" % list(t))

t2 = ("a", "b", "c", "d")
print("t2 is %r" % (t2,))

combs = combinations(t2, 2)
print("List of combinations of t2: %r" % list(combs))
print("List of combinations of t2: %r" % list(combs))

The output is (unexpectedly for me)

t is (1, 2, 3, 4)
list(t) is [1, 2, 3, 4]
list(t) is [1, 2, 3, 4]
t2 is ('a', 'b', 'c', 'd')
List of combinations of t2: [('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'c'), ('b', 'd'), ('c', 'd')]
List of combinations of t2: []

So clearly, list() has side-effects. As expected, converting a tuple to a list does not change the original data, I can do it multiple times. But when I try the same with an iterable returned from combinations, this works only once and then the iterable is seemingly invalid. Does list call next on the iterable so that after it is done, the iterator is at the end or why does this happen? And how can I avoid it?

Upvotes: 1

Views: 63

Answers (2)

Torsten Marek
Torsten Marek

Reputation: 86502

As you have correctly observed, list is destructive, as a generator can only be exhausted once. A simple solution is to use itertools.tee:

>>> c1, c2 = itertools.tee(itertools.combinations(["a", "b", "c"], 2))
>>> print(list(c1))
... will print the entire sequence of combinations
>>> print(list(c2))
... same as before

This may be more memory-conservative as holding on to the entire list, as itertools.tee only needs to hold on to the elements that have not been consumed by all iterators.

Upvotes: 0

TigerhawkT3
TigerhawkT3

Reputation: 49318

itertools.combinations produces a lazy generator, not a complete data structure that gets saved in memory. Once you exhaust it (iterate through it) with something like list(), it is... well, exhausted. Empty. If you want to use it repeatedly, save a reference:

combs = list(combinations(t2, 2))
print("List of combinations of t2: %r" % combs)
print("List of combinations of t2: %r" % combs)

Upvotes: 3

Related Questions