CODEWITHSUNDEEP
pythonpypy
Madarco
Madarco

Reputation: 2114

Pypy JIT gives a speed boost even if a script is run only once?

I have a script that loads several hundred images, resize them and then composes a bigger image

Every time is started with a different set of images:

python myscript.py imageFolder/

Running it in a virtualenv with Pypy doesn't show a noticeable speed gain (all run in ~8 seconds with mprofile, with the pypy version spending more time in PIL.resize and less in packages initialization).

It is because the JIT gives advantage only for long running processes?

If so I can convert the script to a daemon (but I fear of memory leaks).

Upvotes: 2

Views: 673

Answers (2)

NPE
NPE

Reputation: 500437

From your description it appears that PIL.resize() is the dominant operation. That function is written in C and not in Python. Therefore, I doubt you can expect PyPy to make much of a difference to your script.

If you're looking to speed things up, you could consider parallelizing the loading and resizing of the image across multiple cores. I don't generally recommend using threads in Python, normally suggesting the multiprocessing module instead. However, for this particular task multiple threads might actually be a better fit.

Upvotes: 10

jsbueno
jsbueno

Reputation: 110311

For processing images, it is liklely that the bulk of processing time in your script is spent inside PIL's image processing functions.

Those are written in native code, and already are optimized at full native speed - you won't gain much of moving the Python controler parts (code saying which images to open, and such -- think 10-20 bytes for file names against at least 10000s of bytes in each image body).

If you need more speed in there, forget about trying pypy - you can try parallelization of your code through the multiprocess module, though, if you are using a multicore machine.

Upvotes: 1

Related Questions