CODEWITHSUNDEEP
linuxsshgnu-parallel
user3762200
user3762200

Reputation: 447

How to use different --workdir for different hosts in GNU parallel

I want to execute my script.sh on local machine and on remote machine. However, on local machine the script.sh file is inside ~/dir1/and on remote machine is inside /dir2.

What I tried was: inside ~/dir1/on local machine I run:

  parallel --workdir /dir2 -S remotemachine -S : ./script.sh ::: someargs

On remote machine runs fine, but on local machine parallel also tries to run script.sh inside /dir2

Upvotes: 2

Views: 262

Answers (1)

Ole Tange
Ole Tange

Reputation: 33740

A job does not know on which server it will be run. This is (among other things) due to --retries which will retry the same job, but on a different server.

So GNU Parallel does not have a way to make the --workdir different depending on server (apart from the relative dirs: . and ...).

So what can you do instead?

If one of the dirs only exists on one of the machines, then you can cd to that just before running the script. The cd will fail on the machine which does not have the dir:

parallel -S server1,: 'cd dir-on-both; cd dir-on-one; script.sh' ::: some args

Use a common symlink:

ssh server1 ln -s dir1 link1 
ln -s dir2 link1
parallel --workdir link1/ -S server1,: script.sh ::: some args

If it is only due to the executable, set PATH to include both dirs:

PATH=$PATH:dir1:dir2
parallel --env PATH -S server1,: script.sh ::: some args

Edit 20240422

You can now use --plus:

parallel -S localhost,server1,otheruser@server2 --plus echo host:{host} sshlogin:{sshlogin}

These strings will be replaced by $PARALLEL_SSHHOST and $PARALLEL_SSHLOGIN which the shell will evaluate.

Upvotes: 3

Related Questions