Python 2: what is the best way to obtain information during an exception?

Question

I'm writing an app in Python 2. I'd like to have a global exception handler that can catch any and all uncaught exceptions in the app - both in the main thread and in child threads (the program uses threading) - and produce a nice report that the user can Email to me, including the exception stack trace, file, line number, etc. but also the values of several local and global variables.

Main thread exceptions of course will terminate the program. Child thread exceptions obviously don't kill the app, which is fine (and is what I expect). But in either case I would like these exceptions reported to the user in a clean way so they can alert me with all of the necessary information to work on it.

I was thinking the best way to do this would be to override sys.excepthook, since the only other choice I see is wrapping each and every function inside a huge try/except block, and I'm not even sure if these can be nested (honestly, I haven't tried), so that when I do have ways to handle expected exceptions I can do so gracefully.

Here's basically what I want to be able to do:

1. Produce the stack trace, including the file names and line numbers where available, just as Python would do when it prints an exception prior to an exit (or when the thread terminates)
2. Also, retrieve all variables in the local scope (i.e. the function in which the exception is occurring) as well as all global variables (this could be done easily with globals()) I may also want some other specific variables - example, I tend to make my apps into classes, so I can simply instantiate them, and plus the class can then hold all the app-wide variables, so I'd want those app-wide variables, even though the exception might be occurring inside of some class which is underneath this main app class.
3. Get all of this into a formatted text string, in a variable (I can do this on my own, just need to access the information), and then do something with it - dump it to a file on disk, send it via Email, send it via UDP to a custom server, throw it up in a dialog so the user can copy/paste it, whatever. The goal is just to have it come out as a text string.

I've already done a little experimenting with sys.excepthook but I'm a bit in over my head, because I am not sure how to actually pull all of this information from the exception. I've read a little about the traceback module but it's honestly not making much sense to me - it looks like it refers to traceback "objects" but the traceback object being fed into the excepthook method doesn't have any of the properties or attributes listed in the documentation, so it's clearly a different form of "traceback object".

Some advice or pointing in the right direction for documentation would be greatly appreciated!

Answer 1

def myhook(etype, value, tb):
    print "Exception Type: ", etype
    print "Exception: ", value
    while tb.tb_next:
        tb = tb.tb_next
        frame = tb.tb_frame
        co = frame.f_code
        funcname = co.co_name
        coline = frame.f_lineno
        filename = co.co_filename
        print "Exception: FuncName (%s):: File(%s):: Lineno(%s)" %(funcname, filename, coline) 
        print "Exception: Globals::\n", filter(lambda x: x[0] != '__builtins__', frame.f_globals.items())
        print "Exception: Locals::\n", frame.f_locals
        print '-' * 70  

sys.excepthook = myhook

Example Output:

>>> test2.bb()
Exception Type:  <type 'exceptions.NameError'>
Exception:  global name 'asfsdf' is not defined
Exception: FuncName (bb):: File(C:\Python27\test\test2.py):: Lineno(7)
Exception: Globals::
[('aa', <function aa at 0x024F96B0>), ('bb', <function bb at 0x024F90F0>), ('__file__', 'C:\\Python27\\test\\test2.py'), ('__package__', None), ('__na
me__', 'test2'), ('__doc__', None)]
Exception: Locals::
{}
----------------------------------------------------------------------
Exception: FuncName (aa):: File(C:\Python27\test\test2.py):: Lineno(4)
Exception: Globals::
[('aa', <function aa at 0x024F96B0>), ('bb', <function bb at 0x024F90F0>), ('__file__', 'C:\\Python27\\test\\test2.py'), ('__package__', None), ('__na
me__', 'test2'), ('__doc__', None)]
Exception: Locals::
{}
----------------------------------------------------------------------
>>>


cat test2.py:
====================
#!/usr/bin/env python

def aa():
  print asfsdf

def bb():
  aa()          

Answer 2

the traceback object being fed into the excepthook method doesn't have any of the properties or attributes listed in the documentation, so it's clearly a different form of "traceback object".

I'm not seeing that:

import sys
import traceback

def myhandler(e_class, e_instance, tb_obj):
    print "hello**************\n"
    print e_instance
    traceback.print_tb(tb_obj)
    print

    while tb_obj:
        frame = tb_obj.tb_frame
        print 'locals --->', frame.f_locals
        print 'globals -->', frame.f_globals, "\n"

        tb_obj = tb_obj.tb_next

    print "goodbye************"

sys.excepthook = myhandler

x = 10

def do_stuff():
    y = 20

    def inner():
        z = 30
        1/0

    inner()


do_stuff()




--output:--
hello**************

integer division or modulo by zero
  File "1.py", line 32, in <module>
    do_stuff()
  File "1.py", line 30, in do_stuff
    inner()
  File "1.py", line 28, in inner
    1/0

locals ---> {'__builtins__': <module '__builtin__' (built-in)>, '__file__': '1.py', 'traceback': <module 'traceback' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/traceback.pyc'>, '__package__': None, 'sys': <module 'sys' (built-in)>, 'x': 10, '__name__': '__main__', 'do_stuff': <function do_stuff at 0x2b33f0>, '__doc__': None, 'myhandler': <function myhandler at 0x2b3430>}
globals --> {'__builtins__': <module '__builtin__' (built-in)>, '__file__': '1.py', 'traceback': <module 'traceback' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/traceback.pyc'>, '__package__': None, 'sys': <module 'sys' (built-in)>, 'x': 10, '__name__': '__main__', 'do_stuff': <function do_stuff at 0x2b33f0>, '__doc__': None, 'myhandler': <function myhandler at 0x2b3430>} 

locals ---> {'y': 20, 'inner': <function inner at 0x2b33b0>}
globals --> {'__builtins__': <module '__builtin__' (built-in)>, '__file__': '1.py', 'traceback': <module 'traceback' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/traceback.pyc'>, '__package__': None, 'sys': <module 'sys' (built-in)>, 'x': 10, '__name__': '__main__', 'do_stuff': <function do_stuff at 0x2b33f0>, '__doc__': None, 'myhandler': <function myhandler at 0x2b3430>} 

locals ---> {'z': 30}
globals --> {'__builtins__': <module '__builtin__' (built-in)>, '__file__': '1.py', 'traceback': <module 'traceback' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/traceback.pyc'>, '__package__': None, 'sys': <module 'sys' (built-in)>, 'x': 10, '__name__': '__main__', 'do_stuff': <function do_stuff at 0x2b33f0>, '__doc__': None, 'myhandler': <function myhandler at 0x2b3430>} 

goodbye************

I've read a little about the traceback module but it's honestly not making much sense to me

The traceback module is not the same as a traceback object. The two just happen to share the same name. In the sys.excepthook() function, if you name the parameter variable traceback, don't expect it to have any of the methods listed in the traceback module. Here are the properties of a traceback object:

http://docs.python.org/2/reference/datamodel.html