CODEWITHSUNDEEP
c++linuxunixgdbmpi
M K
M K

Reputation: 69

Auto-loading declined when trying to use gdb with MPI?

I have a program that works serially, but when I try to use openMPI, I get the message 

 mpirun noticed that the job aborted, but has no info as to the process that caused that situation.

For the serial program, I run valgrind by typing 

valgrind --leak-check=yes --tool=memcheck /workspace/MK/cmake/Project/buildagain/bin/Project -PR INPUT/chain.inp`

Since I run the program serially by typing, ./runProj.sh type.txt and following the link http://www.open-mpi.org/faq/?category=debugging#memchecker_run to run MPI with valgrind, I typed

mpirun -np 4 valgrind  /workspace/MK/cmake/Project/buildagain/bin/Project -PR INPUT/chain.inp

However, the output is very short, so I cannot see where the problem is occurring

So I tried using gdb with MPI. I typed

 mpirun -np 2 xterm -e gdb ../cmake/Project/buildagain/bin/Project

I got the following output

 [Thread debugging using libthread_db enabled]
  warning: File "opt/apps/ossw/applications/gcc/gcc-4.7/sl6/lib63/libstdc++.so.6.0.17-gdb.py" auto-loading has been declined by your 'auto-load safe-path' set to ...
To enable execution of this file add ...
line to your configuration file "h1/MK/.gdbinit"
 ....

when I type show auto-load safe-path ,the out put is 

 List of directories from which it is safe to auto-load files is usr/share/gdb/auto-load:/usr/lib/debug:/usr/bin/mono-gdb.py

When I type the below I get the same output as before:

 run -PR INP/Chain.inp -iex "set auto-load safe-path /usr/share/gdb/auto-load"

or

 run -PR INP/Chain.inp -iex "set auto-load safe-path /opt/apps/ossw/applications/gcc/gcc-4.7/sl6/lib63/libstdc++.so.6.0.17-gdb.py"

Can anyone help with this?

Upvotes: 0

Views: 1211

Answers (2)

Rob Latham
Rob Latham

Reputation: 5223

GDB from an Xterm from mpirun is possible, but your mpi implementation might make this harder to accomplish.

Say you've got a hello-mpi you want to debug:

$ mpiexec -np 4 xterm -e 'gdb hello-mpi'

With mpich, this will get me four xterms all running gdb (as you can imagine, it's not terribly practical for more than a few processes.

I just tried this with a recent-ish OpenMPI, and got the same result.

But that isn't your problem, as you already tried that. Your problem is that your libraries are in places GDB doesn't expect to look. Here's how I update my auto-load-safe path:

At the top of ~/.gdbinit I have a few lines:

add-auto-load-safe-path /home/robl/work/mpich/code_toybox
add-auto-load-safe-path /home/robl/work/build/mpich/src/mpi/romio/test
...

This creates a white-list of places where GDB expects to find a .gdbinit file

(by the way, you probably want 'set breakpoint pending on' in there, too)

Upvotes: 2

Otheus
Otheus

Reputation: 1052

I think the problem is your trying to launch gdb from within xterm from within mpirun, which I would say you cannot do. But I never used X with mpi. Nor gdb for that matter.

You can use gdb to attach to the running process. So launch your program normally with mpirun, find the process id and attach to it using gdb <progname> <pid>. If you do this and still get the load error, edit the question, I'll delete this answer, and maybe someone else will help.

If it works, or if it fails because the program terminates before you can attach to it, you need to put a pause-loop at the beginning. Usually the way to handle this is some piece of code at the beginning of the program that pauses the program until you've gone in with the debugger and changed a flag. Make a call to pause and define that function this way:

pause() { 
 int pause;
 struct timeval tv = {1,500000}; /* 1.5 seconds */
 while (pause==1) { select(0,NULL,NULL,NULL,&tv); }` 
}

You could also do an MPI-BARRIER, but perhaps the problem is that MPI_INIT is failing or doing something screwy. If it's ok, you can use the barrier like the sleep function. You can also use the node-id to pause only on the node you want to debug.

Upvotes: 1

Related Questions