CODEWITHSUNDEEP
pythonapache-storm
hugos
hugos

Reputation: 1323

Storm alternative for deploying single node

I'm creating an application and would like it to be scalable using Storm. The problem is, most users will actually run it in a single node.

I don't think it's fair for those users to have to install Storm in a single node just to use the application. Is there a way to make the application only require Storm if the user needs to run it on multiple machines without having to rewrite part of the application? Maybe a library?

I know I could always write part of the code specific to run in a Storm environment and let the users decide, but I'm looking for a more elegant approach. If no solution exists I'm thinking about writing a "proxy library" that would decide weather to use bolts in a Storm fashion or in a single node.

I'm using Python but I can use other language if the solution requires so.

Upvotes: 1

Views: 107

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62350

Hmmm... Your question is kind of genetic and hard to answer. I personally would just go with Storm because:

  1. It keeps the code simpler.
  2. Installing Storm is not too hard.
  3. For non parallel execution you might be able to use Storm LocalCluster which execute Storm programs on a single JVM and thus does not require to setup a cluster. However, be aware that LocalCluster was designed for testing code rather than to be used in production (if I am not wrong -- maybe you should double check other sources to verify). Not sure how stable it runs in the long term (and of course, there is no fault-tolerance -- however, if you implement your own local application or "proxy library", you might have no fault-tolerance either).

Writing a "proxy library" is no bad idea. But do not underestimate the complexity of this approach! It will become quite difficult to built it in a fully compatible way..

Upvotes: 1

Related Questions