CODEWITHSUNDEEP
pythoniterator
Jean-François Fabre
Jean-François Fabre

Reputation: 140168

avoid converting iterator to list whenever possible

Suppose I have a function taking one parameter, an iterable, as input, and I want to iterate more than once on the iterable.

If I write it like this:

def a_function(an_iterable):
    for x in an_iterable:
       print(x)
    for x in an_iterable:
       print(x)

the second loop may be executed or not.

Of course, I could do this to avoid creating an unnecessary list if not needed:

def a_function(an_iterable):
    if any(lambda x : type(an_iterable==x) for x in (range,list,set,dict))):
        # use as-is
        pass
    else:
         an_iterable = list(an_iterable)

    for x in an_iterable:
       print(x)
    for x in an_iterable:
       print(x)

that would work for a lot of common cases, but not the general case.

Is there a clean way to detect if I can iterate many times on my iterable object?

Upvotes: 1

Views: 107

Answers (2)

mhawke
mhawke

Reputation: 87074

You could check whether the object has a __next__() method (Python 3), or a next() method (Python 2). If it does then you might assume that it implements the iterator protocol and its values will therefore not be reusable.

>>> l = [1, 2, 3, 4]
>>> it = iter(l)
>>> hasattr(l, '__next__')
False
>>> hasattr(it, '__next__')
True

If there is a __next__ attribute you should then check that it is callable:

>>> hasattr(l, '__next__') and callable(l.__next__)
False
>>> hasattr(it, '__next__') and callable(it.__next__)
True

Upvotes: 0

Bakuriu
Bakuriu

Reputation: 101929

You can use the collections.abc.Sequence class to see if the iterable is actually a sequence:

>>> from collections.abc import Sequence
>>> isinstance([1,2,3], Sequence)
True
>>> isinstance((1,2,3), Sequence)
True
>>> isinstance(range(10), Sequence)
True
>>> isinstance(iter((1,2,3)), Sequence)
False

This wont work for sets:

>>> isinstance({1,2,3}, Sequence)
False

If you want to include sets and mappings use the collections.abs.Set and collections.abc.Mapping:

>>> isinstance({1,2,3}, (Sequence, Set, Mapping))
True

You may want to create an helper function that converts an iterable to a sequence if needed:

def sequencify(iterable):
    if isinstance(iterable, (Sequence, Set, Mapping)):
        return iterable
    return list(iterable)

And now you can just do:

def a_function(iterable):
    iterable = sequencify(iterable)

    for x in iterable:
        print(x)
    for x in iterable:
        print(x)

A simpler alternative is to check that iterable argument does not have a __next__ method:

>>> hasattr([1,2,3], '__next__')
False
>>> hasattr(iter([1,2,3]), '__next__')
True

This works because well-implemented containers are only iterables and not iterator themselves, so they only have an __iter__ method that returns an iterator which has the __next__ method that advances the iteration.

This would lead to:

def sequencify(iterable):
    if not hasattr(iterable, '__next__'):
        return iterable
    return list(iterable)

The final alternative is the simplest: document the argument as a sequence and not an iterable and let the user be responsible for providing the correct type.

Upvotes: 1

Related Questions