CODEWITHSUNDEEP
.netazureasp.net-web-apiqueueazure-webjobs
essmd
essmd

Reputation: 31

Azure WebJobs with many QueueTrigger bad practice?

Currently i am working on a WebApi project using asp.net. I want to move all the non time critical and long running operations into webjobs to keep my API fast and simple.

For example when creating a specific resource, it is created immediatly (entity status: created/provisioning) and a new queue message will be enqueued. Now the webjob receives the message on the queue and starts with the provisioning task. After the job was successfull/failed the resource/entity will be updated (status: completed/failed).

Nothing special. But now i also want to move parts like materializing statistics, reports ... into jobs that are computed and calculated on the fly for now.

When i move all these tasks that are not critical in time into webjobs in separate queues, for example:

ProvisioningQueue UpdateStatisticsA CreateReportB ...

Is it a bad practice to have so many queues (20-50) at once (they all will be polled/checked all the time). For clarity: a queue per message. Sadly i didnt find any way/example how to create multiple job functions that listen to the same queue but with different messages. Also i didnt find anything good/bad about using many queues. Only examples with 1-3 job functions (image convert, resize. ...).

The azure service bus also sounds good (because of topics). I am not sure if i am going the right or a good way with webjobs and storage queues. All i want is to decouple many non time critical and long running tasks from the webapi.

Thank you all!

Upvotes: 1

Views: 394

Answers (1)

Rob Reagan
Rob Reagan

Reputation: 7686

There's no problem at all with using multiple queues, each with a separate message type that needs to be processed.

There are two approaches for processing messages from these queues: throw all of your queue monitoring methods into a single webjob, or split the monitor for each queue into its own separate webjob. I personally like splitting your queue monitors up into separate webjobs because it gives you the ability to move webjobs to separate Web Apps if needed. For example, assume that you have WebJobs A, B, and C that are monitoring Queues A, B, and C. If A has a consistently heavy workload, and B and C are always lightly loaded, you could move A to a separate Web App on a different App Service Plan, and B and C could share a Web App. This is a sort of microservices approach.

Topics will not be useful in this situation.

Service Bus is a good way to go you need to guarantee at least once and at most once delivery. Also, you can easily have one Service Bus queue that holds different message types (if you want to go this way). You send messages in Service Bus via a BrokeredMessage class instance. You can create a BaseMessage class, subclass BaseMessage for each of your message types, package the messages through a BrokeredMessage by serializing to JSON (or whatever format you want to use), then test to see what subclass you have during the dequeuing operation.

Upvotes: 2

Related Questions