CODEWITHSUNDEEP
pythonmultithreadingsockets
cosmosa
cosmosa

Reputation: 761

Increasing performance on a multithreaded web server

I am building a load generator for a simple Python multi-threaded web server. The load generator takes a user-provided number of threads, starts up a client, connects to server and requests a short file, then disconnects. The server creates a new thread for every new client connection.

What is happening is I am able to instantiate about 100 client threads without any issue. But When I increase it to 1000 or more threads, I get undesirable results. I get the following errors:

What is going on and what is the proper way to handle these errors?

Edit
Clearly, I can just catch these socket errors. Is it appropriate to just catch the above errors, or should they be handled? If they should be handled, how should they be handled? These errors are not infrequent or randomly occurring; they occur consistently whenever my load generator spawns > ~100 threads.

I understand now that [Errno 32] Broken pipe occurs when the receiving side of the socket has been closed, and then the write side tries to write to it. In that case, then the receive side (server-side) is closing prematurely, but I'm not sure how to prevent this from happening or if there is even a way.

Upvotes: 1

Views: 765

Answers (1)

famousgarkin
famousgarkin

Reputation: 14116

That might actually reflect the load limits of the simple multi-threaded web server in question.

CPython, the standard Python implementation, doesn't use multiple cores for multithreading because of it's global interpreter lock (GIL). So compute-bound bound tasks like answering loads of HTTP requests are not going to fare so well.

To improve parallel performance, you can:

  • use multiprocessing, like multiprocessing,
  • use alternative Python implementation without such threading limitations, like Jython,
  • use some alternative approach to concurrency, e.g. microthreads like Greenlet, event loop like Twisted, etc.

Production grade web servers usually make use of both options, using multiple worker processes each running multiple threads.

Upvotes: 2

Related Questions