CODEWITHSUNDEEP
azureazure-logic-apps
tlt
tlt

Reputation: 15221

How to schedule jobs at scale with Azure?

I have a web api application deployed to App Service. I need to be able to set arbitrary number of scheduled jobs (http calls to my web api) with arbitrary due date times (from a few hours to a few months).

The point is that i need to be able to set/edit/cancel them on the fly programatically based on different state of my app and i need to maintain thousands of them.

Is there some recommended way to do it?

Upvotes: 0

Views: 187

Answers (1)

Junilo
Junilo

Reputation: 71

I would persist the jobs into either SQL database or Table Storage (I would call it 'ScheduledJobs').

Then have an Azure Function that will query the ScheduledJobs storage at some interval, say every hour to pickup jobs that should be processed at that point in time.

The jobs that are due for processing can then be written to a queue (I would name it 'jobs-queue').

Then have another Azure Function that will pickup jobs from 'jobs-queue'. This Azure Function has the business logic of how to process each job.

From my experience, an Azure Function can only run up to 10 minutes. The time it will take to process each job should not be longer than this period.

Hope this gives you some idea.

Upvotes: 1

Related Questions