Debugging segmentation fault with backtrace in cpp

Question

In my code (written in cpp) I am getting a segmentation fault with following backtrace:

Got signal 11
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0xa053)[0x55614154e053]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
./build/debug/lock_free_list_test(+0x49bf)[0x5561415489bf]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0xa05e)[0x55614154e05e]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
./build/debug/lock_free_list_test(+0x49bf)[0x5561415489bf]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fad)[0x55614154dfad]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
./build/debug/lock_free_list_test(+0x49bf)[0x5561415489bf]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fc9)[0x55614154dfc9]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
./build/debug/lock_free_list_test(+0x49bf)[0x5561415489bf]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fb5)[0x55614154dfb5]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
./build/debug/lock_free_list_test(+0x49bf)[0x5561415489bf]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0xa022)[0x55614154e022]
./build/debug/lock_free_list_test(+0x90bf)[0x55614154d0bf]
./build/debug/lock_free_list_test(+0x498c)[0x55614154898c]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fb5)[0x55614154dfb5]
./build/debug/lock_free_list_test(+0x90bf)[0x55614154d0bf]
./build/debug/lock_free_list_test(+0x498c)[0x55614154898c]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fb1)[0x55614154dfb1]
./build/debug/lock_free_list_test(+0x90bf)[0x55614154d0bf]
./build/debug/lock_free_list_test(+0x498c)[0x55614154898c]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9ca5)[0x55614154dca5]
./build/debug/lock_free_list_test(+0x9fc9)[0x55614154dfc9]
./build/debug/lock_free_list_test(+0x90bf)[0x55614154d0bf]
./build/debug/lock_free_list_test(+0x498c)[0x55614154898c]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fb5)[0x55614154dfb5]
./build/debug/lock_free_list_test(+0x90bf)[0x55614154d0bf]
./build/debug/lock_free_list_test(+0x498c)[0x55614154898c]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fc1)[0x55614154dfc1]
./build/debug/lock_free_list_test(+0x90bf)[0x55614154d0bf]
./build/debug/lock_free_list_test(+0x498c)[0x55614154898c]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fb5)[0x55614154dfb5]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
./build/debug/lock_free_list_test(+0x49bf)[0x5561415489bf]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
Got signal 11
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0xa053)[0x55614154e053]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
./build/debug/lock_free_list_test(+0x49bf)[0x5561415489bf]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test==== start backtrace ====
(+0x9fb5)[0x55614154dfb5]
/lib/x86_64-linux-gnu/libpthread.so.0./build/debug/lock_free_list_test(+0x9fc9)[0x55614154dfc9]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
(./build/debug/lock_free_list_test+0x(13150+0x)49bf[0x)7ff25eb6f150[0x]
5561415489bf./build/debug/lock_free_list_test]
./build/debug/lock_free_list_test(+0x(9ca2+0x)4789[0x)55614154dca2[0x]
556141548789./build/debug/lock_free_list_test]
(./build/debug/lock_free_list_test+0x9fc9()+0x[0x7c5f55614154dfc9)]
[0x./build/debug/lock_free_list_test55614154bc5f]
(./build/debug/lock_free_list_test+0x9159()+0x[0xca5155614154d159)]
[0x./build/debug/lock_free_list_test556141550a51]
(./build/debug/lock_free_list_test+0x(49bf+0x)ffa8[0x)5561415489bf[0x]
556141553fa8./build/debug/lock_free_list_test]
(./build/debug/lock_free_list_test+0x4789()+0x[0x10ba7556141548789)]
[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0./build/debug/lock_free_list_test((+0x+0x77fc7c5f))[0x[0x7ff25eb637fc55614154bc5f]
]
==== end backtrace ====
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x77fc)[0x7ff25eb637fc]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7ff25e2f3b0f]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10f96)[0x7ff25eb6cf96]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11054)[0x7ff25eb6d054]
/lib/x86_64-linux-gnu/libpthread.so.0(sem_wait+0x64)[0x7ff25eb6d144]
./build/debug/lock_free_list_test(+0x101d9)[0x5561415541d9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0xa053)[0x55614154e053]
./build/debug/lock_free_list_test(+0x9159)[0x55614154d159]
./build/debug/lock_free_list_test(+0x49bf)[0x5561415489bf]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x556141554ba7]
==== end backtrace ====
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0(+0x13150)[0x7ff25eb6f150]
./build/debug/lock_free_list_test(+0x9fb5)[0x55614154dfb5]
./build/debug/lock_free_list_test(+0x90bf)[0x55614154d0bf]
./build/debug/lock_free_list_test(+0x498c)[0x55614154898c]
./build/debug/lock_free_list_test(+0x4789)[0x556141548789]
./build/debug/lock_free_list_test(+0x7c5f)[0x55614154bc5f]
./build/debug/lock_free_list_test(+0xca51)[0x556141550a51]
./build/debug/lock_free_list_test(+0xffa8)[0x556141553fa8]
./build/debug/lock_free_list_test(+0x10ba7)[0x==== start backtrace ====
556141554ba7]
==== start backtrace ====
/lib/x86_64-linux-gnu/libpthread.so.0/lib/x86_64-linux-gnu/libpthread.so.0/lib/x86_64-linux-gnu/libpthread.so.0((+0x+0x77fc13150))[0x[0x7ff25eb637fc(7ff25eb6f150]
+0x]
./build/debug/lock_free_list_test13150)[0x7ff25eb6f150(]
+0x./build/debug/lock_free_list_test9fb5)([0x+0x/lib/x86_64-linux-gnu/libc.so.69fb555614154dfb5]
)./build/debug/lock_free_list_test[0x55614154dfb5]
(+0x./build/debug/lock_free_list_test90bf)[0x55614154d0bf(]
+0x./build/debug/lock_free_list_test90bf)([0x+0x55614154d0bf498c]
)./build/debug/lock_free_list_test[0x(55614154898c(]
+0xclone./build/debug/lock_free_list_test498c+0x)3f[0x)(55614154898c[0x+0x]
7ff25e2f3b0f4789./build/debug/lock_free_list_test]
)(==== end backtrace ====
[0x+0x5561415487894789]
)./build/debug/lock_free_list_test[0x556141548789Segmentation fault (core dumped)

So is there a way to debug this and get on which line segfault occurs? (I could not find any posts like this here, if it's duplicate I will delete this). The code itself is pretty big so typing it here would not be correct I guess, I just wonder if I can debug a code with backtrace.

Answer 1

Consider running valgrind which logs the back trace of any memory related errors and it can used on both C and C++. Setting LD_LIBRARY_PATH, would give the exact line of issue. Try to run one test at a time as valgrind can be time consuming sometimes and it doesn't matter if the build is debug/release. I can see that you're having issues on Linux 64bit but if you've an option of using Windows then Dr. memory would be of good help too.

Answer 2

In my code (written in cpp) I am getting a segmentation fault with following backtrace:

This is (apparently) output from your testing framework.

While it contains quite a bit of useful info, you will get a lot more useful info from a debugger (assuming you've compiled your test for debugging with the -g flag).

gdb build/debug/lock_free_list_test
(gdb) run
... wait for SIGSEGV
(gdb) where        ## should print file and line info leading to the crash
(gdb) info locals  ## should print values of local variables and parameters

If for some reason you can't or don't want to use debugger, you can at least get the function, file and line info from your existing crash (again assuming you've built your test with -g flag):

addr2line -fe build/debug/lock_free_list_test 0xa053 0x9159 ...