CODEWITHSUNDEEP
c#azureazure-functionsazure-blob-storage
user7127000
user7127000

Reputation: 3243

How to "trick" Azure Function into running longer than 10 minutes

Azure Functions have a time limit of 10 minutes. Suppose I have a long-running task such as downloading a file that takes 1 hr to download.

[FunctionName("PerformDownload")]
[return: Queue("download-path-queue")]
public static async Task<string> RunAsync([QueueTrigger("download-url-queue")] string url, TraceWriter log)
{
   string downloadPath = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString);
   log.Info($"Downloading file at url {url} to {downloadPath} ...");
   using (var client = new WebClient())
   {
       await client.DownloadFileAsync(new Uri(url), myLocalFilePath);
   }
   log.Info("Finished!");   
}

Is there any hacky way to make something like this start and then resume in another function before the time limit expires? Or is there a better way altogether to integrate some long task like this into a workflow that uses Azure Functions?

(On a slightly related note, is plain Azure Web Jobs obsolete? I can't find it under Resources.)

Upvotes: 9

Views: 17849

Answers (5)

Carlos Salinas
Carlos Salinas

Reputation: 11

I think the issue is related with the Cold Start state. Here you can find more details about it. https://markheath.net/post/avoiding-azure-functions-cold-starts

What you can do is, create an trigger azure function that "ping" your long running function to keep it "warm"

namespace NewProject
{
 public static class PingTimer
    {
        [FunctionName("PingTimer")]
        public static async Task Run([TimerTrigger("0 */4 * * * *")]TimerInfo myTimer, TraceWriter log)
        {
            // This CRON job executes every 4 minutes

log.Info($"PingTimer function executed at: {DateTime.Now}");

            var client = new HttpClient();
            string url = @"<Azure function URL>";
            var result = await client.GetAsync(new Uri(url));

            log.Info($"PingTimer function executed completed at: {DateTime.Now}");
        }
    }}

Upvotes: 1

Katy Shimizu
Katy Shimizu

Reputation: 1121

Adding for others who might come across this post: Workflows composed of several Azure Functions can be created in code using the Durable Functions extension, which can be used to create orchestration functions that schedule async tasks, shut down, and are reawakened when said async work is complete.

They're not a direct solution for long-running tasks that require an open TCP port, such as downloading a file, (for that, a function running on an App Service Plan has no execution time limit), but it can be used to integrate such tasks into a larger workflow.

Upvotes: 10

Thomas
Thomas

Reputation: 29736

10 minutes (based on the timeout setting in the host.json file) after the last function of your function app has been triggered, the VM running your function app will stop.

To prevent this behavior to happen, you can have an empty Timertrigger function that runs every 5 minutes. it wont cost anything and will keep your app up and running.

Upvotes: 2

Josh Carlisle
Josh Carlisle

Reputation: 597

One additional option, depending on the details of your workload, is to take advantage of Azure Container Instances. You can have your Azure Function spin up a container, process your workload (download your file \ do some processing, etc), and then shut down your container for you. Spin up time is typically a few seconds and you only pay for what you use (no need for a dedicated app service plan or vm instance). More details on ACI here.

Upvotes: 3

David Browne - Microsoft
David Browne - Microsoft

Reputation: 89386

Is there any hacky way to make something like this start and then resume in another function before the time limit expires?

If you are on a Consumption Plan you have no control over how long your Function App runs, and so it would not be reliable to use background threads that continue running after your Function entry point completes.

On an App Service plan you're running on VMs you pay for, so you can configure your Function App to run continuously. Also AFAIK you don't have to have a Function timeout on an App Service Plan, so your main Function entry point can run for as long as you want.

Or is there a better way altogether to integrate some long task like this into a workflow that uses Azure Functions?

Yes. Use Azure Data Factory to copy data into Blob Storage, and then process it. The Data Factory pipeline can call Functions both before and after the copy activity.

Upvotes: 6

Related Questions