screen with mpirun

Question

MPI works fine:

$ mpirun -np 2 -H compute-0-0,compute-0-1 echo 1
1
1

However it does not work when launched via screen:

$ rm -f screenlog.*
$ screen -L mpirun -np 2 -H compute-0-0,compute-0-1 echo 1
[screen is terminating]
$ cat screenlog.0 
mpirun: error while loading shared libraries: libimf.so: cannot open shared object file: No such file or directory

This does not help:

$ rm -f screenlog.*
$ screen -L `which mpirun` -xLD_LIBRARY_PATH -np 2 -H compute-0-0,compute-0-1 echo 1
[screen is terminating]
$ cat screenlog.0 
/share/apps/intel/openmpi/bin/mpirun: error while loading shared libraries: libimf.so: cannot open shared object file: No such file or directory

Answer 1

(Is that a Rocks cluster?)

Apparently you're not getting the right $LD_LIBRARY_PATH for some reason.

This isn't a very clean solution, but it should work:

$ screen -L env LD_LIBRARY_PATH=$LD_LIBRARY_PATH mpirun -np 2 \
  -H compute-0-0,compute-0-1 echo 1

I assume that's what you were trying to do with the -xLD_LIBRARY_PATH option, but I don't see that in the documentation I was able to find.

That's what you were trying to do with the -xLD_LIBRARY_PATH option, but that's only going to pick up the value of $LD_LIBRARY_PATH from within the process invoked by screen (and pass it on to the echo 1 command). By that time it's too late. Also, the man page suggests that there should be a space: -x LD_LIBRARY_PATH; I don't know whether it's required.

Is there some script you're sourcing to get the environment (including $LD_LIBRARY_PATH) for MPI? Is it sourced automatically in your login shell startup, or do you do it manually? You might want to write a small wrapper script that sources the setup script and then invokes a specified command. Something like (not tested):

#!/bin/sh
. /share/apps/intel/openmpi/etc/setup.sh # probably not the right name
exec "$@"

Save it as $HOME/bin/mpienv.sh, then you can use:

$ screen -L mpienv.sh mpirun -np 2 -H compute-0-0,compute-0-1 echo 1