CODEWITHSUNDEEP
listpython-3.x
W. Zhu
W. Zhu

Reputation: 825

Inconsistent results when testing speeds of append and concatenate

I am using Python 3.5.2. I was testing a function that would read words from a file and make them into a list. I made two versions: one with the append method and the other with the concatenate operation. Then, I made another function to test the time they take.

import time

def build1(path):
    seq = []
    fin = open(path)
    for line in fin:
        word = line.strip()
        seq.append(word)
    fin.close()
    return seq

def build2(path):
    seq = []
    fin = open(path)
    for line in fin:
        word = line.strip()
        seq += [word]
    fin.close()
    return seq

def test(f, p):
    start = time.time()
    f(p)
    stop = time.time()
    print('Duration: %fs' %(stop - start))

path = 'C:/Users/neem/Desktop/words.txt'
test(build1, path)
test(build2, path)

I tested it a few times and here is my output.

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.171600s
Duration: 0.156000s

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.171600s
Duration: 0.202800s

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.171600s
Duration: 0.202801s

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.171600s
Duration: 0.202800s

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.171600s
Duration: 0.187200s

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.187200s
Duration: 0.187200s

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.187200s
Duration: 0.140400s

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.187200s
Duration: 0.187200s

C:\Users\neem\AppData\Local\Programs\Python\Python35-32>python Programs/Test.py
Duration: 0.171600s
Duration: 0.187200s

Most of the time, build1 is faster than build2. But sometimes build2 is faster than build1, and sometimes they have equal speed. Why does this happen?

Upvotes: 1

Views: 34

Answers (1)

DeepSpace
DeepSpace

Reputation: 81684

These results are pretty much consistent, considering that your CPU is (most probably) doing other things as well. This fact makes your timings irrelevant since you ran each "test code" a single time.

You should use timeit to perform these tests. It will run the code many times and output only the few fastest results which will give you a better comparison.

def build1():
    seq = []
    fin = open('words.txt')
    for line in fin:
        word = line.strip()
        seq.append(word)
    fin.close()
    return seq

def build2():
    seq = []
    fin = open('words.txt')
    for line in fin:
        word = line.strip()
        seq += [word]
    fin.close()
    return seq


import timeit
print(timeit.Timer(build1).repeat(number=1000))
print(timeit.Timer(build2).repeat(number=1000))

This runs build1 and build2 1000 times and returns the 3 best for each function. These results are more consistent than those you saw, with build1 being faster than build2 almost always. I've used a text file with 1024 lines for these tests.

[0.538437745654897, 0.5233988257550134, 0.523436147804647]
[0.5738405088861467, 0.5879328483021968, 0.574590215558163]

Let's time the correct way to do it:

def fast():
    with open('words.txt') as f:
        return [line.strip() for line in f.readlines()]

import timeit
print(timeit.Timer(fast).repeat(number=1000))
>> [0.48617474650164105, 0.46462499504316523, 0.45692032442191644]

Upvotes: 3

Related Questions