How to debug exception not handled by CLR

Question

I am developing a .NET application which is crashing at seemingly random times. It has references to an unmanaged dll which I suspect is throwing an exception which is unhandled. When the application crashes, I get this message when no debugger is attached (compiled as click-once):

At which point I can attach VS2012 as the debugger and see the call stack:

>   ntdll.dll!77e015de()    Unknown
[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] 
ntdll.dll!77e015de()    Unknown
ntdll.dll!77e8861b()    Unknown
ntdll.dll!77eae656()    Unknown
ntdll.dll!77eae6d3()    Unknown
ntdll.dll!77e573bc()    Unknown
ntdll.dll!77e57261()    Unknown
ntdll.dll!77e3b459()    Unknown
ntdll.dll!77e3b42b()    Unknown
ntdll.dll!77e3b3ce()    Unknown
ntdll.dll!77df0133()    Unknown
msvcr100.dll!71fb0269() Unknown
msvcr100.dll!71fb0146() Unknown
mfc100.dll!6c9e3bef()   Unknown
kernel32.dll!76ba14dd() Unknown
msvcr100.dll!71fb016a() Unknown
mfc100.dll!6cbbb734()   Unknown
DataRayOcx.ocx!095fe026()   Unknown
kernel32.dll!76ba14dd() Unknown
msvcr100.dll!71fb016a() Unknown
mfc100.dll!6cbbb734()   Unknown
mfc100.dll!6c9e3e62()   Unknown
DataRayOcx.ocx!096018c4()   Unknown
DataRayOcx.ocx!09603679()   Unknown
msvcr100.dll!71ffc556() Unknown
msvcr100.dll!71ffc600() Unknown
kernel32.dll!76ba33aa() Unknown
ntdll.dll!77e19ef2()    Unknown
ntdll.dll!77e19ec5()    Unknown

and threads:

Not Flagged     5732    0   Worker Thread   msvcr100.dll thread 
DataRayOcx.ocx!09664787 Highest
Not Flagged     5480    0   Main Thread Main Thread clr.dll!70903e82    Normal
Not Flagged     4408    0   Worker Thread   MG17Comms.dll thread    mfc100.dll!6cb8e160 Normal
Not Flagged >   4428    0   Worker Thread   msvcr100.dll thread msvcr100.dll!71fb0269   Normal
Not Flagged     116 0   RPC Thread  RPC Callback Thread 1714fee0    Normal
Not Flagged     5808    0   Worker Thread   ntdll.dll thread    ntdll.dll!77e01f26  Normal
Not Flagged     5372    0   Worker Thread   clr.dll thread  clr.dll!7082c3a6    Normal
Not Flagged     5112    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     4928    0   Worker Thread   clr.dll thread  clr.dll!7090f1e3    Normal
Not Flagged     1380    0   Worker Thread   clr.dll thread  clr.dll!708219a3    Normal
Not Flagged     1632    0   Worker Thread   OphirLMMeasurement.dll thread   OphirLMMeasurement.dll!6ad4a94d Normal
Not Flagged     4324    0   Worker Thread   MG17Core.dll thread MG17Motor.ocx!10034e20  Normal
Not Flagged     5048    0   Worker Thread   clr.dll thread  nipalu.dll!6459a78a Normal
Not Flagged     5028    0   Worker Thread   clr.dll thread  msvcr110_clr0400.dll!72a551ed   Normal
Not Flagged     5556    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     4708    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     3352    0   Worker Thread   clr.dll thread  nipalu.dll!6459a78a Normal
Not Flagged     5256    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     6032    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     4692    0   Worker Thread   OphirLMMeasurement.dll thread   OphirLMMeasurement.dll!6add537a Normal
Not Flagged     6108    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     1504    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     1108    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     4652    0   Worker Thread   wdapi1031.dll thread    wdapi1031.dll!11513a0b  Normal
Not Flagged     2796    0   Worker Thread   OphirLMMeasurement.dll thread   OphirLMMeasurement.dll!6add5f80 Normal
Not Flagged     1036    0   RPC Thread  RPC Callback Thread ole32.dll!76e7a44e  Normal
Not Flagged     3424    0   Worker Thread   clr.dll thread  clr.dll!708d8274    Normal
Not Flagged     5424    0   Worker Thread   clr.dll thread  clr.dll!708f7bc5    Highest
Not Flagged     504 0   Worker Thread   ntdll.dll thread    ntdll.dll!77e0013d  Normal
Not Flagged     2380    0   Worker Thread   clr.dll thread  clr.dll!70b10aed    Normal
Not Flagged     3060    0   Worker Thread   GdiPlus.dll thread  GdiPlus.dll!7327795b    Normal
Not Flagged     3672    0   Worker Thread   clr.dll thread  System.Windows.Forms.ni.dll!6ddfe8e1    Normal
Not Flagged     5268    0   Worker Thread   msiltcfg.dll thread msiltcfg.dll!7371187a   Normal
Not Flagged     1232    0   Worker Thread   msvcr100.dll thread msvcr100.dll!71fb326f   Normal
Not Flagged     5588    0   Worker Thread   clr.dll thread  clr.dll!7090fee1    Normal
Not Flagged     4080    0   Worker Thread   clr.dll thread  clr.dll!708598cd    Normal
Not Flagged     5380    0   Worker Thread   msvcr100.dll thread DataRayOcx.ocx!095cd1ed Normal
Not Flagged     5328    0   Worker Thread   System.Data.dll thread  System.Data.dll!7140b7fd    Normal
Not Flagged     5744    0   Worker Thread   AS5216.dll thread   AS5216.dll!061d3e74 Normal
Not Flagged     2952    0   Worker Thread   AS5216.dll thread   AS5216.dll!061d01dd Above Normal
Not Flagged     3008    0   Worker Thread   AS5216.dll thread   AS5216.dll!061d8e2e Above Normal
Not Flagged     4728    0   Worker Thread   clr.dll thread  clr.dll!7090f1e3    Normal
Not Flagged     2972    0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     3804    0   Worker Thread   nirpc.dll thread    nirpc.dll!6460546a  Normal
Not Flagged     6064    0   Worker Thread   msvcr90.dll thread  mxsout.dll!1b45be1b Normal
Not Flagged     3728    0   Worker Thread   msvcr90.dll thread  mxsout.dll!1b45bd23 Normal
Not Flagged     768 0   Worker Thread   clr.dll thread  clr.dll!7099e9de    Normal
Not Flagged     6096    0   Worker Thread   clr.dll thread  clr.dll!70827f40    Normal

But I really don't know how to read them. I see that the first line Not Flagged 5732 0 Worker Thread msvcr100.dll thread DataRayOcx.ocx!09664787 Highest has highest priority which leads me to be suspicious of that component.

On occasion I am able to catch an exception and get this message:

'External component has thrown an exception.' in System.Windows.Forms.DispatchMessageW() as System.IntPtr
   at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
   at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
   at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
   at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
   at System.Windows.Forms.Application.Run(Form mainForm)
   at Utilities.StartupForm.Main() in C:\...\Utilities\Forms\StartupForm.vb:line 102

Outside of just guessing what the problem is based on the debugger info, what is a good way to debug this type of exception? We are using 3rd parties' software, so when their components raise exceptions, I need to be able to alert them... but first I need to identify the problem accurately.

Answer 1

The debugger is best to identify this kind of issues - so what you did was quite correct. The only problem was that you had invalid symbols and therefore the call stack did not tell you much.

First thing you need to do, as @Hans pointed, is to configure Microsoft Symbol server in your debuggers (you can do this by setting a _NT_SYMBOL_PATH global system variable). With the symbol server enabled, the debugger (Visual Studio for example) will automatically download .pdb files for the system modules (dll and exe files). PDB files contain instructions for the debugger how to decode raw addresses in the stack. Which will convert your current call stack:

ntdll.dll!77e015de()    Unknown
ntdll.dll!77e8861b()    Unknown
ntdll.dll!77eae656()    Unknown
ntdll.dll!77eae6d3()    Unknown

to something much more meaningful. Scan the stack from top to bottom and you should spot the faulting method (if you have PDB files for you third-party libraries) and the module (dll) that owns it. Additionally you may create a minidump at the moment of fault (using for instance procdump) and send it over for examination to the library owners.