Reputation: 3442
I have a Function App targeting .NET 6.0 that logs this error every 10-20 minutes:
22:29:45Z [Verbose] Handling WorkerErrorEvent for runtime:dotnet-isolated, workerId:dotnet-isolated. Failed with: System.TimeoutException: The operation has timed out. at Microsoft.Azure.WebJobs.Script.Grpc.GrpcWorkerChannel.PendingItem.OnTimeout() in /_/src/WebJobs.Script.Grpc/Channel/GrpcWorkerChannel.cs:line 1767
22:29:45Z [Verbose] Attempting to dispose webhost or jobhost channel for workerId: '########-####-####-#################', runtime: 'dotnet-isolated'
22:29:45Z [Verbose] Disposing language worker channel with id:########-####-####-#################
22:29:45Z [Verbose] Disposed language worker channel with id:########-####-####-#################
22:29:45Z [Verbose] No initialized worker channels for runtime 'dotnet-isolated'. Delaying future invocations
22:29:45Z [Verbose] Restarting worker channel for runtime: 'dotnet-isolated'
22:29:45Z [Verbose] Adding jobhost language worker channel for runtime: dotnet-isolated. workerId:########-####-####-#################
22:29:45Z [Information] Worker process started and initialized.
Instead of the Restarting worker channel for runtime: 'dotnet-isolated'
step, this error is occasionally logged instead:
[Error] Exceeded language worker restart retry count for runtime:dotnet-isolated. Shutting down and proactively recycling the Functions Host to recover
[Verbose] Hosting stopping
[Verbose] Stopping file watchers.
[Verbose] Waiting for RpcFunctionInvocationDispatcher to shutdown
[Verbose] Draining invocations from language worker channel completed. Shutting down 'RpcFunctionInvocationDispatcher'
[Information] Stopping JobHost
[Verbose] Stopping ScriptHost instance '########-####-####-#################'.
[Information] Stopping the listener 'Microsoft.Azure.WebJobs.Extensions.Http.HttpTriggerAttributeBindingProvider+HttpTriggerBinding+NullListener' for function '######'
[Information] Stopped the listener 'Microsoft.Azure.WebJobs.Extensions.Http.HttpTriggerAttributeBindingProvider+HttpTriggerBinding+NullListener' for function '######'
[Information] Job host stopped
[Verbose] Stopped ScriptHost instance '########-####-####-#################'.
[Verbose] Hosting stopped
[Verbose] Disposing ScriptHost.
[Verbose] Initiating background SyncTriggers operation
From the Azure Portal > Diagnose and solve problems
> Availability and Performance
> Functions that are not triggering
> Last 24 hours
:
Exception Type | Total Occurrences | Exception Message |
---|---|---|
System.TimeoutException | 123 | Loading function failed. The operation has timed out. |
Exceeded language worker restart retry count for runtime:dotnet- | 41 | Exceeded language worker restart retry count for runtime:dotnet-isolated. Shutting down and proactively recycling the Functions Host to recover |
System.Net.Sockets.SocketException | 19 | Exception encountered while listening to EventStream. An existing connection was forcibly closed by the remote host. |
The first request after the Function App idles for 20 minutes or more takes around 5 - 10 seconds longer to process, even with the Function App set to Always On
. Any requests directly after that is processed within a second.
Environment Variables:
FUNCTIONS_WORKER_RUNTIME
: dotnet-isolated
FUNCTIONS_EXTENSION_VERSION
: ~4
Upvotes: 0
Views: 343
Reputation: 3442
The errors stopped and the initial requests are no longer delayed after changing Settings
> Environment variables
> FUNCTIONS_WORKER_RUNTIME
from dotnet-isolated
to dotnet
in all Function Apps in the App Service Environment.
Upvotes: 0