CODEWITHSUNDEEP
batch-processingapache-storm
Saurabh
Saurabh

Reputation: 73589

How to stop receiving more tuples at one bolt in storm topology

I have a use-case in one of storm topology where I want to control how many tuples are coming into a bolt for processing. for example, I want to stop receiving more tuples when there are 100 tuples already being processed. Probable this can be controlled by topology.max.spout.pending, but this config is at topology level, how can this be achieved at a bolt level.

Upvotes: 0

Views: 91

Answers (1)

Lawrence Craft
Lawrence Craft

Reputation: 324

I don't think this is possible in Storm out of the box. You could block in your execute method but this obviously isn't ideal as tuples will continue to build up in the executor receive queue.

If you want to process tuples in batches like this, why not look into a microbatching solution like Storm Trident or Spark?

Upvotes: 1

Related Questions