CODEWITHSUNDEEP
javaapache-storm
facha
facha

Reputation: 12522

Amount of executors in Storm

I'm playing around with Storm. This is a topology I'm using:

builder.setSpout("word", new RandomSentenceSpout(), 3);
builder.setBolt("exclaim1", new ExclamationBolt(), 6).shuffleGrouping("word");

I thought Storm would spawn 9 executors (3 spouts + 6 bolts) for this topology, but when I actually run it, I can see 11 executors are running.

What are those 2 extra executors?

Upvotes: 7

Views: 1694

Answers (2)

Majid Hajibaba
Majid Hajibaba

Reputation: 3260

They are acker bolts which are responsible for managing acknowledgment mechanism. There is 2 ackers in your topology and each bolt task is equal to one executor by default.

storm uses acker as a bolt task (executor) and if we don't set Number of ackers, it will run some of them in the topology. if you want to exactly manage the number of executors, use the following :

Config conf = new Config();
conf.setNumAckers(1);

Upvotes: 4

gtaank
gtaank

Reputation: 21

Are you setting numTasks somewhere else in the topology? Sharing the topology creation code shall be helpful to answer. Also, r u running this in local or Cluster mode? Executors are always less then or equal to NumTasks. http://www.michael-noll.com/blog/2012/10/16/understanding-the-parallelism-of-a-storm-topology/ This blog very well explains the parallelism in storm.

Upvotes: 0

Related Questions