CODEWITHSUNDEEP
bashwhile-loopparallel-processing
Matthew Bullock
Matthew Bullock

Reputation: 91

While read loop in parallel

I have the following while loop in a bash script however I would like to run these in parallel (an failing) can anyone point me in to the right direction please?

Thanks! 

while read LINE; do
    RAYID=$(echo "$LINE" | jq -r .rayId)
    LINE="$(echo $LINE | sed 's/\([[:digit:]]\{13\}\)[[:digit:]]\{6\}/\1/g')"
    args=( -XPUT "localhost:9200/els/logs/$RAYID?pipeline=geoip-els" -H "Content-Type: application/json" -d "$LINE" )
    curl "${args[@]}" > /dev/null 2>&1
done <<< "$ELS_LOGS"

Upvotes: 0

Views: 1415

Answers (1)

Matias Barrios
Matias Barrios

Reputation: 5056

** EDITED

Additionally to what @TomFenech stated which is correct, I want to add that it would be also nice if you add wait after done, so the script won't finish its execution, until all tasks are completed.

function doSomething(){

        RAYID=$(echo "$1" | jq -r .rayId  ) 
        LINE="$(echo $1 | sed 's/\([[:digit:]]\{13\}\)[[:digit:]]\{6\}/\1/g'  )" 
        args=( -XPUT "localhost:9200/els/logs/$RAYID?pipeline=geoip-els" -H "Content-Type: application/json" -d "$1"  ) 
        curl "${args[@]}" > /dev/null 2>&1 
}

while read LINE; do
   doSomething $LINE &
done <<< "$ELS_LOGS"
wait

Regards!

Upvotes: 1

Related Questions