CODEWITHSUNDEEP
azure-durable-functionstimeoutexceptionlong-running-processes
Unknown Coder
Unknown Coder

Reputation: 1740

Durable function Timeout

I'm Using Azure Durable Functions .Net Core.

[FunctionName("Calculate")]
public async Task RunOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context, ILogger log)
{
    await context.CallActivityAsync<object>(FuncConstants.Cal1, null);
    await context.CallActivityAsync<object>(FuncConstants.Cal2, null);
    await context.CallActivityAsync<object>(FuncConstants.Cal3, null);
    await context.CallActivityAsync<object>(FuncConstants.Cal4, null);
    await context.CallActivityAsync<object>(FuncConstants.Cal5, null);   
    await context.CallActivityAsync<object>(FuncConstants.Cal6, null);
    await context.CallActivityAsync<object>(FuncConstants.Cal7, null);
    await context.CallActivityAsync<object>(FuncConstants.Cal8, null);
    await context.CallActivityAsync<object>(FuncConstants.Cal9, null);
}

[FunctionName("Starter_Calculation")]
public static async Task<HttpResponseMessage> HttpStart(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestMessage req,
    [DurableClient] IDurableOrchestrationClient starter,
    ILogger log)
{
    string instanceId = await starter.StartNewAsync("Calculate", null);
    return starter.CreateCheckStatusResponse(req, instanceId);
}

Here, each function takes more than 5-6 hours to perform some logical calculations (arithmetic operations) as they deal with more than 27-30 million database records.

I'm getting the error for the first function itself as "Timeout value of 00:30:00 was exceeded by function. In Azure, I'm using Premium App Service Plan, so I can set AlwaysOn.

I have a couple of questions:

  1. Is Azure Function the best fit for such a scenario/long-running operations?
  2. If yes, is there any way I can test it locally without a time-out error?

Please advice.

Upvotes: 3

Views: 8584

Answers (1)

AdaTheDev
AdaTheDev

Reputation: 147234

If you're using a Premium or Dedicated (App Service) plan, the default timeout for a function is 30 minutes - which is what you're hitting.

You can extend it by configuring functionTimeout in host.json.

For either of those plans, there is technically no maximum timeout. Though for Premium, execution is only guaranteed for 60 minutes.

See docs: https://learn.microsoft.com/en-us/azure/azure-functions/functions-host-json#functiontimeout

There are 2 main strategies for avoiding timeouts with durable functions:

  1. increase the timeout via host.json
  2. break the work down into smaller units, and have more activity function calls

The 2nd strategy is valuable - smaller unit of work gives you better recovery/resume experience in event of transient failures. Plus, you can parallelize activity function calls to reduce overall processing time.

To run multiple activity functions in parallel, you can await Task.WhenAll multiple activity calls - check out the example in the Fan Out-Fan In pattern in the docs here: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-cloud-backup?tabs=csharp#e2_backupsitecontent-orchestrator-function

It's not clear from your example, whether your activity functions need to wait for the previous one to complete (i.e. have to chain together serially), or whether they are completely independent and therefore can be done in parallel. But, the power of Durable Functions is that support various patterns and take care of a lot of underlying coordination for you.

Upvotes: 6

Related Questions