CODEWITHSUNDEEP
pythonc++multithreadingsocketspython-2.7
sDoky
sDoky

Reputation: 368

Upping the performance of a python server

I am writing a daemon for which I had to choose between Java and Python - technically, I am in charge of language selection completely, but I narrowed it down to these two. I chose Python 2.7 because I have quite a long history with that one. I am 100% sure I do not want to use any other language. Might consider rewriting it for Python 3, but that's as far as I am willing to go. The server's still in design stage so it is not that expensive to rewrite it for Python 3 but C++, Java or any other languages are out of the question. Now, I know how to write a concurrent server, but my question is:

"Is there a better solution?"

In general, the main thread waits for new connections and if there is one, it spawns a new thread to deal with the client and listens for new ones.

# pseudo-code
import threading, socket

class thread(threading.Thread):
    def __init__(self, socket):
        self.socket = socket
    def run(self):
        deal_with_client()

while True:
    s, fromaddr = sock.accept()
    t = thread(s)
    t.start()

I was thinking about multiprocessing instead of threading but I am not sure this is the right approach. My server's got the potential to reach thousands of clients at one time. Hardware is not a problem, we take into account the fact that a python daemon's gotta take more than well written C++ daemon. Anyone's got any better ideas?

Second question:

"Is Python 3 faster than equivalent Python 2.7?"

Thank you in advance for your answers/suggestions/comments/whatever else.

Upvotes: 0

Views: 78

Answers (1)

John Zwinck
John Zwinck

Reputation: 249652

It depends what work you will do for each client. Many servers don't do a lot of CPU-intensive work for each client request, in which case it's better to use an event-driven model like Python Twisted. This is especially true in Python because threading is not its strong suit.

If you have a lot of CPU-intensive work to do, you should use a thread pool (not thread-per-request or thread-per-client, but thread-per-core). And then you can implement the CPU-intensive parts in C or C++ or whatever, and avoid holding the GIL during that time. This will let you unlock the power of multicore with Python.

Finally, no, Python 3 is not likely to be faster than 2.7.

Upvotes: 3

Related Questions