CODEWITHSUNDEEP
pythonreal-timetornado
barmaleikin
barmaleikin

Reputation: 463

Realtime data processing with Python

I am working on a project which is going to consume data from Twitter Stream API and count certain hashtags. But I have difficulties in understanding what kind architecture I need in my case. Should I use Tornado or is there more suitable frameworks for this?

Upvotes: 2

Views: 1372

Answers (1)

chris
chris

Reputation: 63

It really depends on what you want to do with the Tweets. Simply reading a stream of Tweets has not been an issue that I've seen. In fact that can be done on an AWS Micro Instance. I even run more advanced regression algorithms on the real-time feed. The scalability problem arises if you try to process a set of historical Tweets. Since Tweets are produced so fast, processing historical Tweets can be very slow. That's when you should try to parallelize.

Upvotes: 1

Related Questions