CODEWITHSUNDEEP
pythonmultithreadingsockets
jab
jab

Reputation: 5823

Handling Incoming Data from Multiple Sockets in Python

Background: I have a current implementation that receives data from about 120 different socket connections in python. In my current implementation, I handle each of these separate socket connections with a dedicated thread for each. Each of these threads parse the data and eventually store it within a shared locked dictionary. These sockets DO NOT have uniform data rates, some sockets get more data than others.

Question: Is this the best way to handle incoming data in python, or does python have a better way on handling multiple sockets per thread?

Upvotes: 2

Views: 546

Answers (3)

Giridhar Pemmasani
Giridhar Pemmasani

Reputation: 86

As mentioned in Asynchronous UDP Socket Reading question from you, asyncoro can be used to process many asynchronous sockets efficiently. Another benefit with asyncoro in your problem is that you don't need to worry about locking shared dictionary, as with asyncoro at most one coroutine is executing at any time and there is no forced preemption.

Upvotes: 1

Amber
Amber

Reputation: 526493

If you're using threads, that's basically the way you'd go about it.

The alternative is to use one of the various asynchronous networking libraries out there, such as Twisted, Tornado, or GEvent.

Upvotes: 2

hasanyasin
hasanyasin

Reputation: 6253

Using an asynchronous approach will make you much happier. For an example of a well-done implementation of this as a well-known application Tornado is perfect. You can easily use Tornado's ioloop for things other than web servers, too.

There are alternative libraries such as gevent; but I believe Tornado is a better place to look at first since it both provides the loop and a web server implemented on top of it as a great example of how to use the loop well.

Upvotes: 3

Related Questions