CODEWITHSUNDEEP
pythongenerator
Martin Caron
Martin Caron

Reputation: 25

Python inception of generators

I'm trying to make a general way to generate all the combinations of multiple ranges or lists, for example,

[range(0, 2), range(2, 5), range(4, 6), range(2, 3)],

which should return a 2x3x2x1 = 12 element list.

[[0, 2, 4, 2],
 [0, 2, 5, 2],
 [0, 3, 4, 2],
 [0, 3, 5, 2],
 [0, 4, 4, 2],
 [0, 4, 5, 2],
 [1, 2, 4, 2],
 [1, 2, 5, 2],
 [1, 3, 4, 2],
 [1, 3, 5, 2],
 [1, 4, 4, 2],
 [1, 4, 5, 2]]

So far, everything is fine here. When I hardcode it, by doing

x = [ ( [a,b] for a in rgs[0] for b in rgs[1] ) ]
x.append( ( a + [b] for a in x[-1] for b in rgs[2]) )
x.append( ( a + [b] for a in x[-1] for b in rgs[3]) )

I get the good result. However, when I attempt to generalize it, by doing

x = [ ( [a,b] for a in rgs[0] for b in rgs[1] ) ]
for i in range(1,len(rgs)-1):
    x.append( ( a + [b] for a in x[-1] for b in rgs[i+1]) )

I obtain a 6-element list:

[[0, 2, 2, 2],
 [0, 3, 2, 2],
 [0, 4, 2, 2],
 [1, 2, 2, 2],
 [1, 3, 2, 2],
 [1, 4, 2, 2]]

Also, what I notice is that all the ranges generated after the first two use the range in rgs[-1] instead of the correct ones. I struggle to understand why this happens, as I beleive that those two code example are identical, except the latter is the more general form for arbitrary large number of ranges.

Upvotes: 1

Views: 82

Answers (2)

Blckknght
Blckknght

Reputation: 104712

Your issue has to do with creating generator expressions in a loop. Generator expressions are implemented as functions, and like functions, they can have "free" variables that they look up in the containing namespaces. Your generator expressions are accessing the i from outside their definition, and because of this, they end up seeing a different i value you expect.

Here's an example that might be easier to understand:

def gen()
   print(i)
   yield 10

x = []
for i in range(3):
    x.append(gen())  # add several generators while `i` has several different values

for g in x:
    list(g)   # consume the generators, so they can print `i`

Here, rather than using the i value for something useful, I've written a generator function that just prints it out. If you run this code, you'll see that all the generators print out 2, since that's the value of i when they finally run (after the first loop ended).

Your situation is a little more subtle, as you're consuming the previous generator as you create the next one, but the general idea is the same. The generator expression loop that you expect to be over rgs[2] is actually over rgs[3] because it's actually being looked up with rgs[i+1] and i increased between the time the generator expression was declared and when it was consumed.

Upvotes: 1

Joe Patten
Joe Patten

Reputation: 1704

You can use itertools.product to output a list of tuples

Input:

import itertools

a= [range(0, 2), range(2, 5), range(4, 6), range(2, 3)]
list(itertools.product(*a))

Output:

[(0, 2, 4, 2),
 (0, 2, 5, 2),
 (0, 3, 4, 2),
 (0, 3, 5, 2),
 (0, 4, 4, 2),
 (0, 4, 5, 2),
 (1, 2, 4, 2),
 (1, 2, 5, 2),
 (1, 3, 4, 2),
 (1, 3, 5, 2),
 (1, 4, 4, 2),
 (1, 4, 5, 2)]

I did not get the same result when running your first code. I had to change it up a bit:

x = [ ( [a,b] for a in rgs[0] for b in rgs[1] ) ]
x.append( ( a + [b] for a in x[-1] for b in rgs[2]) )
x =  list( a + [b] for a in x[-1] for b in rgs[3])

Most people that don't know about itertools would have done it this way:

x=[]
for i0 in rgs[0]:
    for i1 in rgs[1]:
        for i2 in rgs[2]:
            for i3 in rgs[3]:
                x.append([i0,i1,i2,i3])

Or using a list comprehension (DON'T DO THIS, it is VERY messy looking):

[[i0,i1,i2,i3] for i3 in rgs[3] for i2 in rgs[2] for i1 in rgs[1] for i0 in rgs[0]]

Upvotes: 2

Related Questions