CODEWITHSUNDEEP
pythonpython-itertools
user3248533
user3248533

Reputation: 67

itertools product blocks, when using generators as parameters

I need to construct the cartesian product of output generated by two or more generator functions on the fly.
I would like for itertools.product not to block, but instead to give mr the first product elements before the generator input functions raised StopIteration.
Is there a function that provides sth like that?

I wrote a simple program to demonstrate my problem:

#!/usr/bin/python
import time
import itertools

def g_delay(l, delay):
    for i in range(l):
        yield i
        time.sleep(delay)

def g(l):
    for i in range(l):
        yield i

if __name__ == "__main__":
    start_time = time.time()
    p = itertools.product(g_delay(2,1), g_delay(3,1))
    elapsed_time = time.time() - start_time
    print '%f' % elapsed_time
    for i in p:
        print i

    print

    start_time = time.time()
    p = itertools.product(g(2), g(3))
    elapsed_time = time.time() - start_time
    print '%f' % elapsed_time
    for i in p:
        print i

And the output: 

5.004710
(0, 0)
(0, 1)
(0, 2)
(1, 0)
(1, 1)
(1, 2)

0.000017
(0, 0)
(0, 1)
(0, 2)
(1, 0)
(1, 1)
(1, 2)

What I would like to have as a result is, that the first delay of 5.004710 seconds is similar to the latter one (0.000017) and the blocking occurs on accessing the product elements (in the for loop).

Upvotes: 1

Views: 392

Answers (1)

Helmut Grohne
Helmut Grohne

Reputation: 6778

Here is a version of product (for just two iterators), that tries to be as lazy as possible.

def product(i1, i2):
    for a in i1: 
        j2, i2 = itertools.tee(i2)
        for b in j2:
            yield (a, b)

Upvotes: 1

Related Questions