CODEWITHSUNDEEP
apache-stormapache-storm-topology
Nikita Chernetsov
Nikita Chernetsov

Reputation: 11

ApacheStorm who calls Spout.nextTuple()?

My question is: who specifically calls nextTuple() method in the Spout of Apache Storm topology?

In some sources, it is written that Storm himself does this, but this is not specific and unclear.

Under what conditions does Storm cause this method? Scheduled?

Is there any way to influence this process, say, slow it down?

Upvotes: 1

Views: 788

Answers (1)

Stig Rohde Døssing
Stig Rohde Døssing

Reputation: 3651

It is called here https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/executor/spout/SpoutExecutor.java#L158 (Storm 2.0, for 1.x it is some clojure code that does something similar).

Yes, Storm calls nextTuple. More precisely it is called by the spout executor thread. The method I linked is called in a loop. If the topology.max.spout.pending has been reached, Storm skips calling nextTuple.

Yes, you can slow down how often nextTuple is called when nextTuple doesn't emit anything, in order to avoid wasting CPU if there's nothing to emit. Note the line here https://github.com/apache/storm/blob/master/storm-client/src/jvm/org/apache/storm/executor/spout/SpoutExecutor.java#L176 where a wait strategy is called. The wait strategy is set with the topology.spout.wait.strategy parameter in your topology Config.

The default wait strategy sleeps for 1 ms if there's nothing to emit. You can sleep for longer by configuring topology.sleep.spout.wait.strategy.time.ms. Don't set the wait too high, since processing acks/fails happens in the same thread as nextTuple.

Upvotes: 3

Related Questions