CODEWITHSUNDEEP
pythondebuggingbash
nkrkv
nkrkv

Reputation: 7098

Highlighting python stack traces

I'm working on quite complex project and time after time I have to narrow down problems looking at stack traces. They happen to be very long and involve “my” code, standard library code and 3rd party libraries code at same time. Most of time the real problem is in “my” code and locating it instantly in a stack trace is a bit hard for eyes. Under “my” code I mean the code that is under current working directory.

So I realized that I want something that will colorize stack traces and highlight lines that are mine. Compare original to highlighted.

I could write a python script that I could use this way:

nosetests | colorize_stack_trace.py

But I believe there is a quicker and more elegant way to do this using Linux toolset. Any ideas?

UPD:

Using supercat suggested by Dennis Williamson, the intermediate result is following bash function:

pyst() {
    rc=/tmp/spcrc;
    echo '#################### ### # # # ########################################' > $rc;
    echo '                     blk   0 r ^(.*)$' >> $rc;
    echo '                     mag b 0 r ^\s*File "'`pwd`'/(.*)"' >> $rc;
    spc -c $rc;
}

Now I can do:

nosetests 2>&1 | pyst

Not too elegant, but works at some degree. There are two problems left:

  1. I can't see any output before nosetests completes. I.e. I don't see the progress.
  2. I have to write 2>&1 over and over again.

UPD 2:

Asking this question I had mainly nosetests in mind. And I just found great solution: rednose nose plugin. It highlights paths that are local plus do many more handy readability things.

Returning to original question: problems that I noted with supercat don't relate to it completely but its a problem of Unix shell streaming, flushing, piping, redirecting. So as as an answer to the question asked I accept an answer that suggests supercat.

Upvotes: 29

Views: 5420

Answers (9)

codeananda
codeananda

Reputation: 1320

Another option is to use better_exceptions. This gives you colours and also much more detailed stack trace info. This is preferable to other libraries such as colored-traceback as it's still under active development. Copying over info from their README.

pip install better_exceptions

Set the BETTER_EXCEPTIONS environment variable to any value:

export BETTER_EXCEPTIONS=1  # Linux / OSX
setx BETTER_EXCEPTIONS 1    # Windows

and you're good to go!

Results in outputs like this

enter image description here

Upvotes: 0

tux21b
tux21b

Reputation: 94739

Actually, there is a great Python syntax highlighting library called Pygments, which is also able to highlight tracebacks.

So, all you have to do is:

$ easy_install pygments # downloads and installs pygments
$ cat traceback.txt | pygmentize -l pytb

"pytb" is the shortcut for the PythonTracebackLexer. There is also a special lexer for Python 3 Tracebacks included, which is called "py3tb".

You can format the output in various formats (including html, latex, svg, several image formats and so on). But there is also a terminal formatter available (and if you are wondering... of course there are different color themes available!).

You can use -f html to select another formatter (in that case, the HTML formatter).

Upvotes: 28

kolypto
kolypto

Reputation: 35453

Here's one using ipython:

import sys
from IPython.core.ultratb import ColorTB
print(''.join(ColorTB().structured_traceback(*sys.exc_info())))

Upvotes: 1

Rotareti
Rotareti

Reputation: 53923

There is a nice module just for that purpose:

You just have to download and install it via pip:

pip install colored-traceback

Import it into a top-level file of your project, for exmaple like this:

if DEBUG:
    import colored_traceback
    colored_traceback.add_hook()

And you get a traceback like that for every underling file (colors vary):

Traceback (most recent call last):
  File "./workflowy.py", line 525, in <module>
    main()
  File "./workflowy.py", line 37, in main
    projects = cli.load_json(args, input_is_pipe)
  File "./workflowy.py", line 153, in load_json
    return json.load(sys.stdin)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Upvotes: 7

sirex
sirex

Reputation: 4911

Also you can open traceback file with vim, using :cfile command. Then you can open highlighted ant interactive list of files in traceback using :copen command and jump between these files using set of related vim commands.

Upvotes: 0

Dennis Williamson
Dennis Williamson

Reputation: 360395

Take a look at Supercat (spc). It does both ANSI and HTML highlighting and can be configured for your particular output. It comes with some configuration files for source code files in C and Python, for example and log files, Changelogs, diffs and others.

Based on Dave Kirby's suggestion for vim, this does something similar:

less -p regex file_name

Or

some_command | less -p regex

Upvotes: 6

Dave Kirby
Dave Kirby

Reputation: 26572

Load the text into vim:

nosetests | vim -

Set vim to highlight all lines that match a search

:set hlsearch

Search for lines with "your" path

/.*/path/to/my/code.*

Voila - all the lines with your path will be highlighted.

If you want to highlight the next line as well then you can do that too:

/.*/path/to/my/code.*\n.*

Upvotes: -1

mjjohnson
mjjohnson

Reputation: 1360

As a starting point for colorizing (and otherwise formatting) text, you'll probably want to look at the curses library. Also see this how-to, which looks useful.

As for overriding Python's built-in error handling for all programs...I've never tried it, but I would think that would involve some fairly low-level changes. You could always wrap your code in a huge try/except block, but I assume you don't want to do that. I favor the more Unixy approach of writing a little script that does one thing, and does it well: have it take an input and, if it's a stack trace, colorize it. Otherwise, pass the text through unchanged. Using a pipe, like you suggested, may be the best way. (In this case, to pipe stderr you'd want to do something like this, which merges stderr with stdout before piping: cmd1 2>&1 | cmd2 )

Upvotes: 0

nikow
nikow

Reputation: 21568

Maybe you could use the cgitb module (short official doc here) as a starting point (it creates nice HTML tracebacks). It should be relatively simple to make the modifications you need (e.g. add a color tag according to the file path). But of course this is only viable if you are willing to use the browser to view the traces.

Upvotes: 1

Related Questions