Reputation: 905
What is the best way to scale a Worker Role that is processing many long running Azure Service Bus messages using the QueueClient Message Pump.
If using QueueClient.OnMessageOptions.MaxConcurrentCalls = 6 and QueueClient.OnMessage does that mean i can only process a max of 6 messages at a time?
Is it bad form to have the long running processing within the OnMessage callback to spawn a new Task to complete it's processing?
Should i be using the QueueClient.OnMessageAsync instead?
Thanks for any help.
Upvotes: 4
Views: 912
Reputation: 623
By “long running” do you mean IO-bound or CPU-bound?
Assuming IO-bound then I wouldn’t spawn a new Task in the OnMessage callback. This creates thread management overhead that can slow processing down at scale.
Consider using OnMessageAsync if you are using IO-bound operations and make sure that you await the asynchronous implementations of any of these operations. This uses your existing threads much more efficiently.
If your operations are CPU-bound then Task creation may do more for you. The mechanics of this are discussed in a series of excellent posts by Stephen Cleary:
http://blog.stephencleary.com/2013/10/taskrun-etiquette-and-proper-usage.html
The MaxConcurrentCalls property controls the number of concurrent requests to the service bus. Increasing this number has a limited impact if you’re IO-bound and limited by available bandwidth. I would recommend doing a bit of performance testing with the Azure client-side performance counters to get the optimum value for your environment.
Upvotes: 3