CODEWITHSUNDEEP
azure-web-app-service
hultqvist
hultqvist

Reputation: 18409

Distribute work when scaling out Azure App Service

How can I distribute tasks over multiple scale-out instances of an Azure App Service?

The tasks will read data from one queue each, aggregate over one minute, write the result into a blob storage.

My goals for the tasks are these:

My main concern is not work load performance but responsiveness/uptime. A single instance could host all tasks but if that instance would restart there would be an unwanted downtime.

Upvotes: 1

Views: 483

Answers (1)

Rob Reagan
Rob Reagan

Reputation: 7686

From my understanding of your requirements, I don't think you want one task running per queue. This would introduce a point of failure if your task went down.

Instead, consider using continuous WebJobs. You could place all of your WebJobs on a single Web App in your App Service Plan, then scale the Web App to two or more instances. So if you had WebJobs A, B, and C monitoring Queues 1, 2, and 3 and then spun up a second Web App instance, you'd have WebJobs A1 and A2 monitoring Queue 1. If one goes down, the other picks up the slack while the other instance is recovering.

Also, I recommend using Service Bus Queues if you require that a message be delivered at least once and no more than once. Check out the ServiceBusTrigger for WebJobs. Note that when a particular WebJob instance dequeues a Service Bus Queue message, the message is locked so that it is not processed twice.

To decrease response time (if your queue backs up), you can just add more instances of your Web App. You can also set the auto-scale of your Web App to be dependent on queue length, so you can scale up and scale down automatically depending on if your queue starts to back up.

Upvotes: 2

Related Questions