CODEWITHSUNDEEP
pythonpython-3.xfile-iopython-2.x
Jim Green
Jim Green

Reputation: 31

Why python3 is slower than python2 in file writing

Same code separately running on Python3.6 and Python2.7 on the same machine, the program is just to write file in a loop, the result is python3.6 is much slower than python2.7 surprisingly. So why is that? @ubuntu18.04

# python3.6
import time
tt1 = time.time()

with open('test1.txt', 'w') as fout:
    for i in range(1000000):
        print(1, file=fout)
print (time.time()-tt1)

Output: 0.6536719799041748

# python2.7 
import time

tt1 = time.time()


with open('test1.txt', 'w') as fout:

    for i in xrange(1000000):
        print >> fout, 1

print(time.time() - tt1)

Output: 0.1888

Upvotes: 3

Views: 103

Answers (1)

ShadowRanger
ShadowRanger

Reputation: 155458

There are at least two major differences in behavior here:

  1. open handles changed from being bytes oriented (only newline translation is handled, and even that only adds overhead on Windows) to text oriented; both prints convert the values to be printed to a str, but the Python 3 str supports the entire Unicode range and has to be encoded in the locale encoding to write it to a file opened for write in text mode; Python 2 writes the bytes raw.
  2. print changed from a statement (with direct bytecode support) to a function call (with no special interpreter support). Function calls are higher overhead, especially when arguments are passed by keyword, and you're doing enough of these that the costs would be expected to add up.

I'll note that for any moderately advanced code, use of print is pretty uncommon; it's an extremely high overhead function. You might try replacing it with just fout.write('1\n') and avoiding a ton of overhead.

Upvotes: 5

Related Questions