CODEWITHSUNDEEP
c++gdbshared-librariesstack-trace
John Jumper
John Jumper

Reputation: 481

Find source line in shared library from stack trace

I have a long-running program that had a (probably intermittent) segmentation fault after 9 hours, and all I have is a stack trace. I would like to find the source line where the segmentation fault occurred.

The segfault occurred in a C++ module (that uses OpenMP) that was called from Python using ctypes. The C++ module was compiled with debugging symbols on linux. The Python program is itself running under OpenMPI, making debugging even more challenging, and I do not have a core dump.

The top of the stack trace is below. I am interested in finding out whatever I can about the source line _ZN13BackbonePairs13compute_valueE11ComputeMode+0x5e1. This is clearly my BackbonePairs::compute_value function, but that function does not have a thread launch in it, so I am unsure about the libpthread in the stack trace.

[midway2-0015:35095] *** Process received signal ***
[midway2-0015:35095] Signal: Segmentation fault (11)
[midway2-0015:35095] Signal code: Invalid permissions (2)
[midway2-0015:35095] Failing at address: 0x7fd7633c6000
[midway2-0015:35095] [ 0] /lib64/libpthread.so.0(+0xf100)[0x7fdb86986100]
[midway2-0015:35095] [ 1] /home/jumper/upside/py/../obj/libupside.so(_ZN13BackbonePairs13compute_valueE11ComputeMode+0x5e1)[0x7fdb7328c571]
[midway2-0015:35095] [ 2] /home/jumper/upside/py/../obj/libupside.so(_ZN11DerivEngine7computeE11ComputeMode+0x4c9)[0x7fdb73277a59]
[midway2-0015:35095] [ 3] /home/jumper/upside/py/../obj/libupside.so(evaluate_energy+0x65)[0x7fdb73202ba5]
[midway2-0015:35095] [ 4] /software/python_ucs4-2.7.13-el7-x86_64+gcc-6.2/lib/python2.7/lib-dynload/_ctypes.so(ffi_call_unix64+0x4c)[0x7fdb5280e15a]
[midway2-0015:35095] [ 5] /software/python_ucs4-2.7.13-el7-x86_64+gcc-6.2/lib/python2.7/lib-dynload/_ctypes.so(ffi_call+0x153)[0x7fdb5280cd33]
[midway2-0015:35095] [ 6] /software/python_ucs4-2.7.13-el7-x86_64+gcc-6.2/lib/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x277)[0x7fdb528042b7]
[midway2-0015:35095] [ 7] /software/python_ucs4-2.7.13-el7-x86_64+gcc-6.2/lib/python2.7/lib-dynload/_ctypes.so(+0x9b52)[0x7fdb527fab52]
[midway2-0015:35095] [ 8] /software/python_ucs4-2.7.13-el7-x86_64+gcc-6.2/lib/libpython2.7.so.1.0(PyObject_Call+0x43)[0x7fdb86be79a3]
[midway2-0015:35095] [ 9] /software/python_ucs4-2.7.13-el7-x86_64+gcc-6.2/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x55ed)[0x7fdb86c9df7d]

Upvotes: 0

Views: 1098

Answers (1)

Gert Wollny
Gert Wollny

Reputation: 529

When you load the library into gdb, the command

disas /m

should show the disassembly with the offsets interleaved with the source code lines (if this information is available). cf: gdb machine code.

Pthread in the stack trace might show up if you call some MPI code that got in-lined by the compiler (e.g. a mutex lock/unlock).

Upvotes: 1

Related Questions