CODEWITHSUNDEEP
apache-stormscalability
ctimus
ctimus

Reputation: 101

Apache Storm - Capability for dynamic scaling

Wanted to check the feasibility, thoughts and best practices around this topic while at the same time would try to add more information as I find along.

Apache Storm can be designed, configured and managed statically (as per the usage that I have seen) where we can manage the parallelism, throttling etc. based on the project inflow, throughput etc. Also we can monitor the lag, utilization etc. to manage the properties to better use the resources or to manage scaling up/down.

The problem here is, these all need to be done statically and with manual intervention.

Is there a way that Storm can be configured or deployed into a container based environment where we can get the auto scaling with less manual check / static configuration.

Upvotes: 0

Views: 690

Answers (1)

Stig Rohde Døssing
Stig Rohde Døssing

Reputation: 3651

There is work going on to launch Storm workers in Docker here https://issues.apache.org/jira/browse/STORM-3271. I am not aware of any autoscaling work, but Storm can be scaled up and down without having to redeploy the topology using the storm rebalance command.

If you have a suggestion for how this would work, consider posting to the Storm dev mailing list

Upvotes: 1

Related Questions