CODEWITHSUNDEEP
pythonperformanceinterpreterprofile
vkris
vkris

Reputation: 2165

Avoid python setup time

This image below says python takes lot of time in user space. Is it possible to reduce this time at all ?

In the sense I will be running a script several 100 times. Is it possible to start python so that it takes time to initialize once and doesn't do it the subsequent time ??

enter image description here

Upvotes: 3

Views: 882

Answers (3)

Arne Babenhauserheide
Arne Babenhauserheide

Reputation: 2513

I just searched for the same and found this:

http://blogs.gnome.org/johan/2007/01/18/introducing-python-launcher/

Python-launcher does not solve the problem directly, but it points into an interesting direction: If you create a small daemon which you can contact via the shell to fork a new instance, you might be able to get rid of your startup time.

For example get the python-launcher and socat¹ and do the following:

PYTHONPATH="../lib.linux-x86_64-2.7/" python python-launcher-daemon &
echo pass > 1
for i in {1..100}; do 
    echo 1 | socat STDIN UNIX-CONNECT:/tmp/python-launcher-daemon.socket & 
done

Todo: Adapt it to your program, remove the GTK stuff. Note the & at the end: Closing the socket connection seems to be slow.

The essential trick is to just create a server which opens a socket. Then it reads all the data from the socket. Once it has the data, it forks like the following:

        pid = os.fork()
        if pid:
            return

        signal.signal(signal.SIGPIPE, signal.SIG_DFL)
        signal.signal(signal.SIGCHLD, signal.SIG_DFL)

        glob = dict(__name__="__main__")
        print 'launching', program
        execfile(program, glob, glob)

        raise SystemExit

Running 100 programs that way took just 0.7 seconds for me.

You might have to switch from forking to just executing the code instead of forking if you want to be really fast.

(That’s what I also do with emacsclient… My emacs takes ~30s to start (due to excessive use of additional libraries I added), but emacsclient -c shows up almost instantly.)

¹: http://www.socat.org

Upvotes: 3

John Machin
John Machin

Reputation: 82924

Write the "do this several 100 times" logic in your Python script. Call it ONCE from that other language.

Upvotes: 1

user2665694
user2665694

Reputation:

Use timeit instead:

http://docs.python.org/library/timeit.html

Upvotes: 0

Related Questions