CODEWITHSUNDEEP
asp.netazureazure-web-app-service
Kevin Krueger
Kevin Krueger

Reputation: 669

Azure web app worker process recycling unexpectedly

We have an ASP.NET MVC web site running in an Azure Standard Medium App Service plan.

Occasionally the worker process is recycled without warning.

I added code to Application_End in global.asax.cs to log the HostingEnvironment.ShutdownReason.

The logged reason is HostingEnvironment which as I understand it means the hosting environment initiated the shutdown.

There are no application errors or events logged. "Always On" in Application Settings is set to "On".

How can I determine the root cause of the shutdown?

Upvotes: 1

Views: 2802

Answers (3)

Kevin Krueger
Kevin Krueger

Reputation: 669

After adding more detailed shutdown reason logging as described here, I found the following details logged:

"Shutdown message: Change Notification for critical directories.
File Change Notification Error in App_LocalResources
HostingEnvironment initiated shutdown
Change Notification for critical directories.
File Change Notification Error in App_LocalResources
Change Notification for critical directories.
File Change Notification Error in App_LocalResources
Change Notification for critical directories.
File Change Notification Error in App_LocalResources
Change Notification for critical directories.
File Change Notification Error in App_LocalResources
Change Notification for critical directories.
File Change Notification Error in App_LocalResources
Change Notification for critical directories.
File Change Notification Error in App_LocalResources
Change Notification for critical directories.
File Change Notification Error in App_LocalResources
CONFIG change
CONFIG change

This leads me to believe that there are some transient issues happening in Azure causing ASP.NET change notification to fail, thus triggering the application to restart. A similar issue is described in this question.

As a work-around I have added fcnMode = "disabled" to the httpRuntime element in the web.config. The root cause remains unknown. The accepted answer in the linked question indicates the cause in that case was an SMB bug in the Northern Europe data center. There must be other causes, because I experienced this issue in the US East datacenter almost 3 years later.

Upvotes: 0

Jeff Nall
Jeff Nall

Reputation: 44

Any chance you've found any 500.121 errors in your web logs? I experienced an issue with a deadlock inside of the w3wp process that resulted in zero I/O for awhile. It turns out azure slams the door on any processes with zero I/O for about 4 minutes.

The entries in the log will jump out at you. Most of the W3C properties normally listed were empty (including the client IP!)

Ended up being a code bug.

Upvotes: 0

Christopher Thomas
Christopher Thomas

Reputation: 372

I'm not too familiar with Azure so perhaps this isn't relevant but by default IIS recycles app pools every 29 hours unless you specify otherwise.

some more info here.

Upvotes: 1

Related Questions