CODEWITHSUNDEEP
linuxbashshellscpgnu-parallel
flash
flash

Reputation: 1519

exit shell script from subshell if files are not present on remote server or disk is full?

I am using below script to copy files in parallel from remote server:

#!/bin/bash

primary=/bat/primary
secondary=/bat/secondary

do_copy() {
  el=$1
  primsec=$2
  (scp -C -o StrictHostKeyChecking=no \
   goldy@"$remote":"$pod1"/abc_187_"$el"_111_8.data "$primsec"/. > /dev/null 2>&1)\
  || \
  (scp -C -o StrictHostKeyChecking=no \
   goldy@"$remote":"$pod2"/abc_187_"$el"_111_8.data "$primsec"/. > /dev/null 2>&1)\
  || \
  (scp -C -o StrictHostKeyChecking=no \
   goldy@"$remote":"$pod3"/abc_187_"$el"_111_8.data "$primsec"/. > /dev/null 2>&1)\
  || \
  (scp -C -o StrictHostKeyChecking=no \
   goldy@"$remote":"$pod4"/abc_187_"$el"_111_8.data "$primsec"/. > /dev/null 2>&1)
}
export -f do_copy

parallel -j "$5" do_copy {} $primary ::: ${pri[@]} &
parallel -j "$5" do_copy {} $secondary ::: ${snd[@]} &
wait
echo "all files copied"

On the remote server I have four folders represented by $pod1, $pod2, $pod3, $pod4. If files are not there in $pod1, then it should be there in $pod2, or it will be there in $pod3 or it will be there in $pod4 folder.

Now what I want to do is:

I tried adding exit command next to last scp command but it just exits from the subshell not from whole script. Is there any way to do these things?

Upvotes: 0

Views: 101

Answers (1)

Ole Tange
Ole Tange

Reputation: 33685

I will assume that the file exists on only one pod, so trying all pods is not a problem.

#!/bin/bash

primary=/bat/primary
secondary=/bat/secondary

# Activate parset and env_parallel if not already done
. `which env_parallel.bash`

# Make array with $primary as many times as elements in $pri
parset prim -N0 echo $primary ::: ${pri[@]} 
# Make array with $secondary as many times as elements in $pri
parset seco -N0 echo $secondary ::: ${sec[@]}
export remote

do_copy() {
  el=$1
  primsec=$2
  pod=$3
  scp -C -o StrictHostKeyChecking=no \
   goldy@"$remote":"$pod"/abc_187_"$el"_111_8.data "$primsec"/
}
export -f do_copy
copy_one_file() {
  parallel do_copy $1 $2 ::: pod{1..4}
  if [ $? == 4 ] ; then
    return 1
  else
    return 0
  fi
}
export -f copy_one_file
parallel --halt now,fail=1 copy_one_file :::  ${pri[@]} ${sec[@]} :::+ ${prim[@]} ${seco[@]}

parset makes an array with $primary as long as $pri.

:::+ links the elements from $pri to the elements in $prim.

You will need parallel version > 20170422 for parset.

Upvotes: 1

Related Questions