How do you use gdb-multiarch to debug a Yocto generated application core dump?

Question

I have an application, compiled for ARM using a Yocto build system, that is crashing and generating a core dump.

On the target, I have installed GDB and my application dbg ipk and I can load the core dump file in GDB and view the stack trace without issue.

I want to be able to analyse this core dump without a target. I've tried using gdb-multiarch but cannot see the contents of the stack trace.

I have access to both the installed binary and the debug symbols from the dbg ipk. When I load the binary gdb-multiarch logs that it has loaded the symbols from the debug symbol file but I cannot get valid symbols in the stack trace.

I don't need stack symbols for all the shared libraries, just from my application.

Interestingly if I run gdb list, I can see code from my main.cpp file but not at the point of the crash.

# gdb-multiarch
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".

(gdb) file my_application
Reading symbols from my_application...
Reading symbols from .debug/my_application...

(gdb) core core-dump-file
warning: Can't open file /lib/libc-2.21.so during file-backed mapping note processing
warning: Can't open file /lib/libgcc_s.so.1 during file-backed mapping note processing
warning: Can't open file /lib/libm-2.21.so during file-backed mapping note processing
warning: Can't open file /lib/libstdc++.so.6.0.21 during file-backed mapping note processing
warning: Can't open file /usr/lib/libboost_system.so.1.60.0 during file-backed mapping note processing
warning: Can't open file /usr/lib/libboost_filesystem.so.1.60.0 during file-backed mapping note processing
warning: Can't open file /usr/lib/libcrypto.so.1.0.0 during file-backed mapping note processing
warning: Can't open file /usr/lib/libssl.so.1.0.0 during file-backed mapping note processing
warning: Can't open file /lib/librt-2.21.so during file-backed mapping note processing
warning: Can't open file /usr/lib/libjsoncpp.so.1.8.4 during file-backed mapping note processing
warning: Can't open file /lib/libpthread-2.21.so during file-backed mapping note processing
warning: Can't open file /lib/libatomic.so.1.1.0 during file-backed mapping note processing
warning: Can't open file /lib/libdl-2.21.so during file-backed mapping note processing
warning: Can't open file /lib/ld-2.21.so during file-backed mapping note processing
[New LWP 9334]
[New LWP 9344]
[New LWP 9346]
[New LWP 9347]
[New LWP 9348]
warning: Could not load shared library symbols for 14 libraries, e.g. /lib/libdl.so.2.
Use the "info sharedlibrary" command to see the complete listing.
Do you need "set solib-search-path" or "set sysroot"?
Core was generated by `/usr/sbin/my_application'.
Program terminated with signal SIGABRT, Aborted.
#0  0xb6aa0996 in ?? ()
[Current thread is 1 (LWP 9334)]

(gdb) bt
#0  0xb6aa0996 in ?? ()
#1  0xb6aae448 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

(gdb) list
warning: Source file is more recent than executable.
13      #include ....
14      ......
..... shows code from the main.cpp in my application
15      ......
(gdb)

Is there any way to see the contents of the stack trace using gdb-multiarch?

Answer 1

You should try setting up a sysroot so that GDB can find all of those libraries that it's trying to load.

Create a directory like /tmp/sysroot and then copy all of those files from the target, so you'll end up with:

/tmp/sysroot/lib/libc-2.21.so
/tmp/sysroot/lib/libgcc_s.so.1
... etc ...

Then in GDB:

(gdb) set sysroot /tmp/sysroot
(gdb) file my_application
(gdb) core core-dump-file

With luck GDB should now load all the libraries from the sysroot, and hopefully the backtrace symbols will be within one of those libaries.