CODEWITHSUNDEEP
parallel-processingapache-storm
Jagrut Dalwadi
Jagrut Dalwadi

Reputation: 337

Best Storm Parallelism from various examples

I am applying parallelism for my storm topology. I have set number of worker node=1.

Example#1 I am setting number of Task and number of executor for particular component as "2".

Example#2: no of tasks < no of executors I am setting number of Tasks as "1" and number of executor as "2" for particular component.

Example#3: no of tasks > no of executors I am setting number of Tasks as "5" and number of executor as "1" for particular component.

I am not getting which of the above example will lead to Best parallelism for topology and suggest which one gives benefits of Storm Parallelism? Please help me to understand this.

Upvotes: 0

Views: 232

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62330

Did you read this article? https://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html

To get good performance, you should set the number of executors to the number of available cores (each executors runs in an own thread). Using more tasks than executors is only beneficial if you want to change the parallelism during runtime.

Your "example#2" is no valid configuration: #tasks >= #executors must always be true (otherwise, there would be threads with no work).

Upvotes: 1

Related Questions