Reputation: 115
I have a C# function app (.net Core 3.1) that processes items from a queue. I need to slow down the processing as there is another external API that I need to talk to and it is rate limited.
I am using
await Task.Delay(1000);
to pause the function app for a period of time between calls to the API. In my local development environment it works perfectly and pauses for the time period given. When deployed to Azure it would appear that the delay is not occurring and processing of the queue items proceeds much quicker than expected.
I have tried this with a long period of 10000ms (10 seconds) and can see the delay when testing locally but when deployed to Azure the queue items are used up reasonably quickly and certainly less 10 seconds apart.
The function declaration looks like this:
[FunctionName("ProcessUserQueue")]
public async Task RunAsync([QueueTrigger(QueueNames.ProcessUserQueueName)]string myQueueItem, ILogger log)
{
log.LogInformation($"ProcessUserQueue trigger function processed: {myQueueItem}");
await _userService.ProcessUserAsync(myQueueItem, log);
}
Inside of ProcessUserAsync the first thing I do is call
await Task.Delay(10000);
and it works, but only in local dev environment.
My host.json looks like this (with a queue batch size of one):
{
"version": "2.0",
"functionTimeout": "00:10:00",
"extensions": {
"queues": {
"batchSize": 1,
"maxDequeueCount": 5
}
}
}
Any ideas as to why this is not working, or how to better limit the speed of API calls within the function?
Upvotes: 0
Views: 3423
Reputation: 325
I don't think adding delay to the HttpTrigger function while reading the Queue is not going to work in a real-world scenario as the function will timeout if the processing of the items took longer than the function timeout period.
For operations that can take longer than the HttpTrigger function timeout time, use Azure Durable Functions.
Durable Functions provides durable timers for use in orchestrator functions to implement delays or to set up timeouts on async actions.
[FunctionName("BillingIssuer")]
public static async Task Run(
[OrchestrationTrigger] IDurableOrchestrationContext context)
{
for (int i = 0; i < 10; i++)
{
DateTime deadline = context.CurrentUtcDateTime.Add(TimeSpan.FromDays(1));
await context.CreateTimer(deadline, CancellationToken.None);
await context.CallActivityAsync("SendBillingEvent");
}
}
For implementing delays in Azure Durable function using Timers, see article: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-timers?tabs=csharp
For Azure Durable functions see: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=csharp
Upvotes: 0
Reputation: 4786
There are multiple ways to delay the process:
Instead of using Task.Delay()
you can use Thread.Sleep()
.
Task.Delay() is used for a logical delay without blocking the current thread.
Thread.Sleep() is used for blocking the current thread.
You can use durable functions
to achieve the process delay Reference
it consists of setting an initialVisibilityDelay
: Add message
For initialVisibilityDelay you need to add a temporary queue and then add a function.
Refer for more info
Upvotes: 1