CODEWITHSUNDEEP
python
alicederyn
alicederyn

Reputation: 13247

How to make a repeating generator in Python

How do you make a repeating generator, like xrange, in Python? For instance, if I do:

>>> m = xrange(5)
>>> print list(m)
>>> print list(m)

I get the same result both times — the numbers 0..4. However, if I try the same with yield:

>>> def myxrange(n):
...   i = 0
...   while i < n:
...     yield i
...     i += 1
>>> m = myxrange(5)
>>> print list(m)
>>> print list(m)

The second time I try to iterate over m, I get nothing back — an empty list.

Is there a simple way to create a repeating generator like xrange with yield, or generator comprehensions? I found a workaround on a Python tracker issue, which uses a decorator to transform a generator into an iterator. This restarts every time you start using it, even if you didn't use all the values last time through, just like xrange. I also came up with my own decorator, based on the same idea, which actually returns a generator, but one which can restart after throwing a StopIteration exception:

@decorator.decorator
def eternal(genfunc, *args, **kwargs):
  class _iterable:
    iter = None
    def __iter__(self): return self
    def next(self, *nargs, **nkwargs):
      self.iter = self.iter or genfunc(*args, **kwargs):
      try:
        return self.iter.next(*nargs, **nkwargs)
      except StopIteration:
        self.iter = None
        raise
  return _iterable()

Is there a better way to solve the problem, using only yield and/or generator comprehensions? Or something built into Python? So I don't need to roll my own classes and decorators?

Update

The comment by u0b34a0f6ae nailed the source of my misunderstanding:

xrange(5) does not return an iterator, it creates an xrange object. xrange objects can be iterated, just like dictionaries, more than once.

My "eternal" function was barking up the wrong tree entirely, by acting like an iterator/generator (__iter__ returns self) rather than like a collection/xrange (__iter__ returns a new iterator).

Upvotes: 25

Views: 19220

Answers (6)

pylang
pylang

Reputation: 44485

You can reset iterators with more_itertools.seekable, a third-party tool.

Install via > pip install more_itertools.

import more_itertools as mit


def myxrange(n):
    """Yield integers."""
    i = 0
    while i < n:
        yield i
        i += 1

m = mit.seekable(myxrange(5))
print(list(m))
m.seek(0)                                              # reset iterator
print(list(m))
# [0, 1, 2, 3, 4]
# [0, 1, 2, 3, 4]

Note: memory consumption grows while advancing an iterator, so be wary wrapping large iterables.

Upvotes: 0

Matt S
Matt S

Reputation: 1454

Using itertools its super easy. 

import itertools

alist = [1,2,3]
repeatingGenerator = itertools.cycle(alist)

print(next(generatorInstance)) #=> yields 1
print(next(generatorInstance)) #=> yields 2
print(next(generatorInstance)) #=> yields 3
print(next(generatorInstance)) #=> yields 1 again!

Upvotes: 15

John Millikin
John Millikin

Reputation: 200776

Not directly. Part of the flexibility that allows generators to be used for implementing co-routines, resource management, etc, is that they are always one-shot. Once run, a generator cannot be re-run. You would have to create a new generator object.

However, you can create your own class which overrides __iter__(). It will act like a reusable generator:

def multigen(gen_func):
    class _multigen(object):
        def __init__(self, *args, **kwargs):
            self.__args = args
            self.__kwargs = kwargs
        def __iter__(self):
            return gen_func(*self.__args, **self.__kwargs)
    return _multigen

@multigen
def myxrange(n):
   i = 0
   while i < n:
     yield i
     i += 1
m = myxrange(5)
print list(m)
print list(m)

Upvotes: 21

SmartElectron
SmartElectron

Reputation: 1441

use this solution:

>>> myxrange_ = lambda x: myxrange(x)
>>> print list(myxrange_(5))
... [0, 1, 2, 3, 4]
>>> print list(myxrange_(5))
... [0, 1, 2, 3, 4]

>>> for number in myxrange_(5):
...     print number
... 
    0
    1
    2
    3
    4
>>>

and with a decorator:

>>> def decorator(generator):
...     return lambda x: generator(x)
...
>>> @decorator
>>> def myxrange(n):
...   i = 0
...   while i < n:
...     yield i
...     i += 1
...
>>> print list(myxrange(5))
... [0, 1, 2, 3, 4]
>>> print list(myxrange(5))
... [0, 1, 2, 3, 4]
>>>

Simple.

Upvotes: -1

Beni Cherniavsky-Paskin
Beni Cherniavsky-Paskin

Reputation: 10039

If you write a lot of these, John Millikin's answer is the cleanest it gets.

But if you don't mind adding 3 lines and some indentation, you can do it without a custom decorator. This composes 2 tricks:

  1. [Generally useful:] You can easily make a class iterable without implementing .next() - just use a generator for __iter__(self)!

  2. Instead of bothering with a constructor, you can define a one-off class inside a function.

=>

def myxrange(n):
    class Iterable(object):
        def __iter__(self):
            i = 0
            while i < n:
                yield i
                i += 1
    return Iterable()

Small print: I didn't test performance, spawning classes like this might be wasteful. But awesome ;-)

Upvotes: 2

Omnifarious
Omnifarious

Reputation: 56048

I think the answer to that is "No". I'm possibly wrong. It may be that with some of the funky new things you can do with generators in 2.6 involving arguments and exception handling that would allow something like what you want. But those features are mostly intended for implementing semi-continuations.

Why do you want to not have your own classes or decorators? And why did you want to create a decorator that returned a generator instead of a class instance?

Upvotes: 0

Related Questions