CODEWITHSUNDEEP
pythonperformance
Arief
Arief

Reputation: 379

generator is slower compared to list comprehension

From what I know, we may use generator when we want to use the values only once. Using the 2 examples below, my logic is that the 2nd one should be faster, because the first one creates a list first and then loop over the values.. while the 2nd only process the values from the generator. Yet, when I calculate the time, the list comprehension is always faster then the generator. Why is this?

1st:

x = []
a = time.perf_counter()
for j in [i**2 for i in range(20000)]:
    x.append(j)
print( time.perf_counter() - a )

2nd:

x = []
a = time.perf_counter()
for j in (i**2 for i in range(20000)):
    x.append(j)
print( time.perf_counter() - a )

Upvotes: 5

Views: 653

Answers (1)

AKX
AKX

Reputation: 169416

Yeah, generators and genexprs are generally (heh) slower than list comprehensions, but on the other hand they're lazily evaluated, and you don't have to pay the memory cost for a fully precomputed list either. I imagine the speed difference is caused by call frame overhead with the (implicit or explicit) yield throwing values around.

Using your code, but timeit to measure it, and with a third version using a generator function:

import timeit

def f1():
    x = []
    for j in [i**2 for i in range(20000)]:
        x.append(j)
    return x

def f2():
    x = []
    for j in (i**2 for i in range(20000)):
        x.append(j)
    return x


def f3():
    def gen():
        for i in range(20000):
            yield i ** 2
    x = []
    for j in gen():
        x.append(j)
    return x


print(timeit.timeit(f1, number=100))
print(timeit.timeit(f2, number=100))
print(timeit.timeit(f3, number=100))

The results (Python 3.7.0) seem to point to genexprs being exactly as fast as generator functions, about 4-5% slower than the list comprehension.

f1 = 2.882695159
f2 = 3.0303254170000002
f3 = 3.002670741

Upvotes: 8

Related Questions