Passing exception stack traces from C++ to Python (via Cython)

Question

Problem

I have a logging and infra system which I can't (and don't want to modify), in Python, that relies on the Tracebacks of the exceptions.

I have C++ code wrapped with Cython. This C++ code can potentially raise exceptions (std::runtime_errors). Cython gives the ability to translate exception (e.g. with except +). However, and that makes sense of course, the translation only translates the exception type and message.

In my case, I would want to translate the stack trace in the C++ exception (which I can recover to a string), into a Python's Traceback. Obviously, the translation is not fully meaningful, but it allows me to "cheat" and continue (or replace) the stack trace in Python with something that represents the stack trace in C++.

Assuming I can parse the C++ exception's stack trace into meaningful values, I wonder what's the best way to achieve that. I have one idea listed below, and I will be happy to get your opinions on it or share alternatives.

I'll add that this whole thing feels very flaky, but I'm both curious about it and want to avoid doing stuff like putting the C++ stack trace in the Python's exception message.

My Idea

So after looking a bit, I found that CPython has a method that is called _PyTraceback_Add. Specifically it's signature is void _PyTraceback_Add(const char *funcname, const char *filename, int lineno) and looks like it adds the values (funcname, filename and lineno) into the current trace. Link to implementation: https://github.com/python/cpython/blob/66f77caca39ba39ebe1e4a95dba6d19b20d51951/Python/traceback.c#L257

My thoughts were to use that right when I raise the Python exception in Cython. Then, I could use the exception string from C++ and insert the values into the existing trace.

However, this method seems to be private in CPython and I'm not sure it will be sound to rely on it in production code.

Are there any good alternatives to this solution?

Thanks!

Answer 1

Well the function seems to be using only public API's, so can just do the same steps instead.

With that said, it looks unlikely to change anytime soon.

Also, it seems somebody's already doing exactly what you're doing: see issue #24743.