NZJames
NZJames

Reputation: 5055

Nancy Server sporadically crashes with NullReferenceException

We have a service that seems to die from time to time, at the moment it seems to be daily but happened less frequently in the past, though still a problem. It seems to be the server with the most load, none of our other servers experience this problem. The exception we get it

2018-03-14 11:06:09,574 [9] FATAL Topshelf.Runtime.Windows.WindowsServiceHost The service threw an unhandled exception
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Net.HttpListener.EndGetContext(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Owin.Host.HttpListener.OwinHttpListener.<ProcessRequestsAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

With literally no other information about what part of our code was being called at this time. It seems like quite a low level Nancy issue so I was just hoping someone else had any thoughts or had come across this before?

The service can be restarted after failure and works as normal, so seems a transient problem.

Upvotes: 0

Views: 146

Answers (1)

superkeci
superkeci

Reputation: 171

As far as I see, the exception does not point any place in Nancy code, but most probably before invoking Nancy handlers. It might be a problem with your OWIN configuration or some other handler that runs before Nancy.

My experience tells:

  • Check your Owin and Nancy pipes, especially dynamic objects those are created and destroyed runtime,
  • Also see your operating system's event logs - there might be something else prematurely kills your services,
  • Try to run a profiler and gather detailed information on runtime behavior - focus on memory usage
  • Try to enhance your logging, at least try to define boundries of owin middlewares including nancy.

That's all I can say by just seeing a simple stack trace. Good luck!

Upvotes: 1

Related Questions