CODEWITHSUNDEEP
bashshellmultiprocessing
user8147819
user8147819

Reputation: 41

wait not working in shell script?

I am running a for loop in which a command is run in background using &. In the end i want all commands to return value..

Here is the code i tried 

for((i=0 ;i<3;i++)) {

    // curl command which returns a value   & 
    } 
    wait

// next piece of code

I want to get all three returned value and then proceed.. But the wait command does not wait for background processes to complete and runs the next part of code. I need the returned values to proceed..

Upvotes: 3

Views: 7194

Answers (2)

Petr Skocik
Petr Skocik

Reputation: 60056

Shell builtins have documentation accessible with help BUILTIN_NAME. help wait yields:

wait: wait [-n] [id ...]
    Wait for job completion and return exit status.

    Waits for each process identified by an ID, which may be a process ID or a
    job specification, and reports its termination status.  If ID is not
    given, waits for all currently active child processes, and the return
    status is zero.  If ID is a a job specification, waits for all processes
    in that job's pipeline.

    If the -n option is supplied, waits for the next job to terminate and
    returns its exit status.

    Exit Status:
    Returns the status of the last ID; fails if ID is invalid or an invalid
    option is given.

which implies that to get the return statuses, you need to save the pid and then wait on each pid, using wait $THE_PID.

Example:

sl() { sleep $1; echo $1; return $(($1+42));  }
pids=(); for((i=0;i<3;i++)); do sl $i & pids+=($!); done; 
for pid in ${pids[@]}; do wait $pid; echo ret=$?; done

Example output:

0
ret=42
1
ret=43
2
ret=44

Edit:

With curl, don't forget to pass -f (--fail) to make sure the process will fail if the HTTP request did:

CURL Example:

#!/bin/bash
URIs=(
    https://pastebin.com/raw/w36QWU3D 
    https://pastebin.com/raw/NONEXISTENT
    https://pastebin.com/raw/M9znaBB2
     )

pids=(); for((i=0;i<3;i++)); do
    curl -fL "${URIs[$i]}" &>/dev/null &
    pids+=($!)

done
for pid in "${pids[@]}"; do
    wait $pid
    echo ret=$?
done

CURL Example output:

ret=0
ret=22
ret=0

Upvotes: 4

Mark Setchell
Mark Setchell

Reputation: 207365

GNU Parallel is a great way to do high-latency things like curl in parallel.

parallel curl --head {} ::: www.google.com www.hp.com www.ibm.com

Or, filtering results:

parallel curl --head -s {} ::: www.google.com www.hp.com www.ibm.com | grep '^HTTP'
HTTP/1.1 302 Found
HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently

Here is another example:

parallel -k 'echo -n Starting {} ...; sleep 5; echo done.'  ::: 1 2 3 4
Starting 1 ...done.
Starting 2 ...done.
Starting 3 ...done.
Starting 4 ...done.

Upvotes: 0

Related Questions