CODEWITHSUNDEEP
pythonlistloopslist-comprehension
Sercan
Sercan

Reputation: 2169

How does list-comprehension "create list" and "append elements" different than simple loop?

While I was reading and searching about what exactly causes performance difference between loops and list comprehension (based on below simple test cases, list comprehension faster), I encountered following posts here in SO:

Simple test cases:

def f1():
    t = []
    for i in range(10000):
        t.append(i)

def f2():
    t = [i for i in range(10000)]

Why is a list comprehension so much faster than appending to a list?

Are list-comprehensions and functional functions faster than “for loops”?

What I understand from above posts that main differences as follows;

  1. For loop builds a list but, list comprehension does not
  2. For loop loads append method on each iteration but, list comprehension does not

Then I used disassembler to see the details and I saw the below steps for following block of code:

Code:

def f2():
    t = [i for i in range(10000)]
    
dis.dis(f2)

Disassembler result:

0 BUILD_LIST              
2 LOAD_FAST                
4 FOR_ITER                 
6 STORE_FAST               
8 LOAD_FAST                
10 LIST_APPEND        
12 JUMP_ABSOLUTE            
14 RETURN_VALUE

Based on Python doc; 0 BUILD_LIST creates list and 10 LIST_APPEND uses append method by contrast with above related posts:

LIST_APPEND(i)
    Calls list.append(TOS[-i], TOS). Used to implement list comprehensions.

BUILD_LIST(count)
    Works as BUILD_TUPLE, but creates a list.

I could not figure out what I am missing here. Is the way list comprehension builds and appends different than for loop, because anyway LIST_APPEND contains append method and BUILD_LIST creates a list? or maybe the reason for the performance difference is something else? Can someone please clarify this for me?

Thanks a lot!

Upvotes: 2

Views: 912

Answers (1)

Yam Mesicka
Yam Mesicka

Reputation: 6601

I've tried a different approach:

from collections import Counter


def f1():
    t = []
    for i in range(10000):
        t.append(i)

def f2():
    t = [i for i in range(10000)]

    
f1i = Counter(i.opname for i in dis.get_instructions(f1))
f2i = Counter(i.opname for i in dis.get_instructions(f2))

print(f"Only in regular append: {f1i - f2i}")
print(f"Only in list comprehension: {f2i - f1i}")

The results are (Python 3.7.6):

Only in regular append: Counter({'LOAD_FAST': 2, 'BUILD_LIST': 1, 'STORE_FAST': 1, 'SETUP_LOOP': 1, 'FOR_ITER': 1, 'LOAD_METHOD': 1, 'CALL_METHOD': 1, 'POP_TOP': 1, 'JUMP_ABSOLUTE': 1, 'POP_BLOCK': 1})
Only in list comprehension: Counter({'LOAD_CONST': 2, 'MAKE_FUNCTION': 1, 'CALL_FUNCTION': 1})

You can see that the "regular" append uses LOAD_METHOD (for list.append), LOAD_FAST, CALL_METHOD and POP_TOP each iteration:

dis.dis(f1)

  5           0 BUILD_LIST               0
              2 STORE_FAST               0 (t)

  6           4 SETUP_LOOP              26 (to 32)
              6 LOAD_GLOBAL              0 (range)
              8 LOAD_CONST               1 (10000)
             10 CALL_FUNCTION            1
             12 GET_ITER
        >>   14 FOR_ITER                14 (to 30)
             16 STORE_FAST               1 (i)

  7          18 LOAD_FAST                0 (t)
             20 LOAD_METHOD              1 (append)
             22 LOAD_FAST                1 (i)
             24 CALL_METHOD              1
             26 POP_TOP
             28 JUMP_ABSOLUTE           14
        >>   30 POP_BLOCK
        >>   32 LOAD_CONST               0 (None)
             34 RETURN_VALUE

It is also recommended to keep in mind that the opcodes change from one version to another.

Upvotes: 2

Related Questions