CODEWITHSUNDEEP
apache-storm
yunfeng.guo
yunfeng.guo

Reputation: 185

How to measure the time taken by storm job

I want to benchmark my topology with different parallelism_hint. In order to test it, i prepare a 1G log file to emit to Storm. The problem is how can i measure the time taken by my topology programmatically

Upvotes: 0

Views: 104

Answers (2)

om-nom-nom
om-nom-nom

Reputation: 62835

time like latency or what? You can see latencies for every component on Storm UI.

enter image description here

Also, there is a "capacity" column -- if you have bottlenecks that you need to scale, their capacity will be close or higher that 1.

This statistics is also can be fetched programatically via Storm REST api.

Upvotes: 1

Matthias J. Sax
Matthias J. Sax

Reputation: 62350

If you know how many tuples your spout is going to emit and if you have fault-tolerance enabled, you can simple count all incoming acks... Basically, take a startup timestamp in Spout.open(...), count the call to Spout.ack() and if the count reaches the input size, take a final timestamp.

Upvotes: 0

Related Questions