CODEWITHSUNDEEP
pythonsocketspublish-subscribe
jhlu87
jhlu87

Reputation: 4019

python best way to publish/receive data between programs

I'm trying to figure out the best way to publish and receive data between separate programs. My ideal setup is to have one program constantly receive market data from an external websocket api and to have multiple other programs use this data. Since this is market data from an exchange, the lower the overhead the better.

My first thoughts were to write out a file and have the others read it, but that seems like there would be file locking issues. Another approach I tried was to use UDP sockets, but it seems like the socket blocks the rest of the program when receiving. I'm pretty new at writing full fledged programs instead of little scripts so sorry if this a dumb question. Any suggestions would be appreciated. Thanks!

Upvotes: 0

Views: 79

Answers (2)

Venkata S S K M Chaitanya
Venkata S S K M Chaitanya

Reputation: 394

You can use SQS, It is easy to use and the Python documentation for it is great. If you want a free one you can use Kafka

Upvotes: 1

Goran B.
Goran B.

Reputation: 580

Try something like an message queue, e.g. https://github.com/kr/beanstalkd, and you essentially control it via the client ... one that collects and sends, and one that consumes and marks what it has read ... and so on.

Beanstalk is super-light-weight and simple compared to other message queues which are more like multi app. systems rather than queues necessarily.

Upvotes: 1

Related Questions