CODEWITHSUNDEEP
linuxmultithreadingbashshellmultiprocessing
TolMera
TolMera

Reputation: 416

Multiprocessing in Shell Script (BASH)

I've written a script and What I'm wanting to do is get it to run as fast as possible by using sub-processing or background-processing or what ever I need to use in order to speed up the computations.

So I was looking at using the & symbol in my script to simply speed up processing wherever I could, but when I put it in here, I don't get the results echo'ed to me.

This is the function that I have written and am trying to improve apon.

InTri(){
    while read line
    do
        V1X=$(echo "$line" | awk '{print $1}') &
        V1Z=$(echo "$line" | awk '{print $2}') &
        V2X=$(echo "$line" | awk '{print $3}') &
        V2Z=$(echo "$line" | awk '{print $4}') &
        V3X=$(echo "$line" | awk '{print $5}') &
        V3Z=$(echo "$line" | awk '{print $6}') &
        run=$(echo "$line" | awk '{print $7}') &
        wait
        echo "$V1X $V1Z $V2X $V2Z $V3X $V3Z $run"
    done < <(mysql -u root -ppassword LightCycle -N -e "SELECT V1X, V1Z, V2X, V2Z, V3X, V3Z, ID FROM Temp3 WHERE (V1X <= $x OR V2X <= $x OR V3X <= $x) AND (V1X >= $x OR V2X >= $x OR V3X >= $x) AND (V1Z <= $z OR V2Z <= $z OR V3Z <= $z) AND (V1Z >= $z OR V2Z >= $z OR V3Z >= $z);")
}

I've read the "man bash" (or what of it I could understand) but I don't understand why this is not working.

Help! :)

Thanks to anyone who can, your input is much appreciated.

Upvotes: 0

Views: 3491

Answers (4)

dogbane
dogbane

Reputation: 274532

You might think that creating more processes will make it faster, but it might actually make it slower because of the overhead of creating new processes and all that piping.

Why not simply do the following?

mysql -u root -ppassword LightCycle -N -e "SELECT V1X, V1Z, V2X, V2Z, V3X, V3Z, ID FROM Temp3 WHERE (V1X <= $x OR V2X <= $x OR V3X <= $x) AND (V1X >= $x OR V2X >= $x OR V3X >= $x) AND (V1Z <= $z OR V2Z <= $z OR V3Z <= $z) AND (V1Z >= $z OR V2Z >= $z OR V3Z >= $z);" | awk '{print $1" "$2" "$3" "$4" "$5" "$6" "$7}'

Or if you want to read them into variables:

while read V1X V1Z V2X V2Z V3X V3Z run
do
  echo "$V1X $V1Z $V2X $V2Z $V3X $V3Z $run"
done < <(mysql -u root -ppassword LightCycle -N -e "SELECT V1X, V1Z, V2X, V2Z, V3X, V3Z, ID FROM Temp3 WHERE (V1X <= $x OR V2X <= $x OR V3X <= $x) AND (V1X >= $x OR V2X >= $x OR V3X >= $x) AND (V1Z <= $z OR V2Z <= $z OR V3Z <= $z) AND (V1Z >= $z OR V2Z >= $z OR V3Z >= $z);")

Upvotes: 2

TolMera
TolMera

Reputation: 416

In order to get the script above working as it should, I just needed to remove the & before the "wait"

InTri(){
    while read line
    do
        V1X=$(echo "$line" | awk '{print $1}') &
        V1Z=$(echo "$line" | awk '{print $2}') &
        V2X=$(echo "$line" | awk '{print $3}') &
        V2Z=$(echo "$line" | awk '{print $4}') &
        V3X=$(echo "$line" | awk '{print $5}') &
        V3Z=$(echo "$line" | awk '{print $6}') &
        run=$(echo "$line" | awk '{print $7}') 
        wait
        echo "$V1X $V1Z $V2X $V2Z $V3X $V3Z $run"
    done < <(mysql -u root -ppassword LightCycle -N -e "SELECT V1X, V1Z, V2X, V2Z, V3X, V3Z, ID FROM Temp3 WHERE (V1X <= $x OR V2X <= $x OR V3X <= $x) AND (V1X >= $x OR V2X >= $x OR V3X >= $x) AND (V1Z <= $z OR V2Z <= $z OR V3Z <= $z) AND (V1Z >= $z OR V2Z >= $z OR V3Z >= $z);")
}

Upvotes: 0

Guru
Guru

Reputation: 16974

If you feel your program is getting slower due to those 7 awk commands, you can do something like this without any awk:

$ line="1 2 3 4 5 6 7"
$ a=($line)
$ echo ${a[0]}
1
$ echo ${a[1]}
2
$ echo ${a[@]}
1 2 3 4 5 6 7

where "a" is an array which contains all the elements of the line variable in it.

Upvotes: 2

Michael Krelin - hacker
Michael Krelin - hacker

Reputation: 143051

Whatever happens in & line happens in a child process, that can't modify parent's environment. You'll probably need to open pipes and read from them in parent process. For tasks with comparable costs of execution with your example it isn't worth the effort.

In this example you can just do

cat <(echo "$line" | awk '{print $1}') <(echo "$line" | awk '{print $1}') ...

Instead of this whole loop body.

Upvotes: 2

Related Questions