CODEWITHSUNDEEP
python
Ken T
Ken T

Reputation: 2553

list comprehension with generator

I used generator to generate some lists. However, it doesn't work like what I expected. I first used numgen1 to to generate the list while it doesn't work properly. And I switched to numgen2 which can gave me what I want properly. But both the numgen1 and numgen2 are basically the same (at least I think), why do they behave so differently? May anyone give me some explanations?

def numgen1(start, end, delta):
    curr=start 
    while curr[1] < end[1] or curr[2]<end[2]:
        yield curr
        curr[2] += delta


print 'Output1: ', [ i for i in numgen1([1,1,1],[1,1,5],1)]

def numgen2(start, end, delta):
    curr=start 
    while curr[1] < end[1] or curr[2]<end[2]:
        yield [curr[0], curr[1], curr[2]]
        curr[2] += delta


print 'Output2: ', [ i for i in numgen2([1,1,1],[1,1,5],1)]

Here are the outputs.

Output1:  [[1, 1, 5], [1, 1, 5], [1, 1, 5], [1, 1, 5]]
Output2:  [[1, 1, 1], [1, 1, 2], [1, 1, 3], [1, 1, 4]]

Follow up question:

After reading the answer from unutbu, I write one more generator that is used to test what unutbu said. But the generator doesn't behave as unutbu said. I am so confused about whether a generator is yielding a pointer or a copy of the value.

def numgen3(start, end, delta):
    curr=start 
    while curr<end:
        yield curr
        curr += delta

print list(numgen3(1,10,1))

Here is the output. [1, 2, 3, 4, 5, 6, 7, 8, 9]

This time I try to generate some numbers instead of some lists. But why aren't all the elements in the list 9? I didn't create a new number, I just yield the same number (curr). I expect the result of numgen3 should be similar to that of numgen1.

Upvotes: 1

Views: 133

Answers (1)

unutbu
unutbu

Reputation: 879511

curr is a list. curr[2] += delta is modifying the list in-place.

When you yield curr, you are yielding the same list over and over. When you print Output1 you are seeing this same list being printed many times.

When you yield [curr[0], curr[1], curr[2]] you are generating a new list. Thus, when you print Output2, you see different values.

Notice how modifying curr affects all items in result, because result is a list containing 3 items, each the same list curr:

curr = [0,0,0]
result = [curr for i in range(3)]
print(result)
# [[0, 0, 0], [0, 0, 0], [0, 0, 0]]

curr[2] = 100
print(result)
# [[0, 0, 100], [0, 0, 100], [0, 0, 100]]

You could "fix" numgen1 by yielding list(curr) since list(curr) returns a new list with the same elements as in curr (i.e. a "shallow copy"):

def numgen1(start, end, delta):
    curr=start 
    while curr[1] < end[1] or curr[2]<end[2]:
        yield list(curr)
        curr[2] += delta

print 'Output1: ', [ i for i in numgen1([1,1,1],[1,1,5],1)]

yields

Output1:  [[1, 1, 1], [1, 1, 2], [1, 1, 3], [1, 1, 4]]

Regarding numgen3:

def numgen3(start, end, delta):
    curr=start 
    while curr<end:
        yield curr
        curr += delta

print list(numgen3(1,10,1))

It helps to make a mental distinction between mutable and immutable values. A list is mutable, numbers such as ints are immutable.

Lists are containers. You can mutate its contents without changing the reference to the container. Thus curr is a list, curr[2] += delta mutates the contents at index 2, but yield curr yields the very same list.

In numgen3, curr is an immutable int. curr += delta assigns curr to a new immutable int. It no longer references the same object. yield curr yields that value. Those different values are accumulated in the list comprehension, and thus you see the result containing different values.

Here is another perspective on what it means to modify a list in-place: The modification is done in-place if the contents of the list change, while the memory address of the list itself does not change.

id(obj) returns the memory address of the object obj. Notice that modifying curr[2] does not change the id of curr:

In [162]: curr = [0,0,0]

In [163]: id(curr)
Out[163]: 196192940

In [164]: curr[2] += 1

In [165]: curr
Out[165]: [0, 0, 1]

In [166]: id(curr)
Out[166]: 196192940

Compare that with what happens when you increment a variable assigned to an int:

In [191]: curr = 1

In [192]: id(curr)
Out[192]: 150597808

In [193]: curr += 1

In [194]: id(curr)
Out[194]: 150597796

Here, curr is not being modified in-place. curr is simply being redirected to reference a new value at a new memory address.

Upvotes: 3

Related Questions