CODEWITHSUNDEEP
qsub
anu
anu

Reputation: 25

qsub Job using GNU parallel not running

I am trying execute qsub job in a multinode(2) and PPN of 20 using GNU parallel, However it shows some error.

#!/bin/bash
#PBS -l nodes=2:ppn=20
#PBS -l walltime=02:00:00
#PBS -N down

cd $PBS_O_WORKDIR
module load gnu-parallel

for cdr in /scratch/data/v/mt/Downscale/*;do
  (cp /scratch/data/v/mt/DWN_FILE_NEW/* $cdr/)
  (cd $cdr && parallel -j20 --sshloginfile $PBS_NODEFILE 'echo {} | ./vari_1st_imge' ::: *.DS0 )
done

When I run the above code I got the following error(Please note all the path are properly checked, and the same code without qsub is running properly in a normal computer)

$ ./down 
parallel: Error: Cannot open echo {} | ./vari_1st_imge.

& for $qsub down -- no output is creating

I am using parallel --version GNU parallel 20140622

Please help to solve the problem

Upvotes: 1

Views: 750

Answers (1)

Ole Tange
Ole Tange

Reputation: 33748

First try adding --dryrun to parallel.

But my feeling is that $PBS_NODEFILE is not set for some reason, and that GNU Parallel tries to read the command as the --sshloginfile.

To test this:

echo $PBS_NODEFILE
(cd $cdr && parallel --sshloginfile $PBS_NODEFILE -j20 'echo {} | ./vari_1st_imge' ::: *.DS0 )

If GNU Parallel now tries to open -j20 then it is clear that it is empty.

Upvotes: 1

Related Questions