Run a shell function in the background (using &) but still retrieve results

Question

Is it possible to call a BASH function with '&' and return a value? Or is there a better way? Basically I want to run the same command several times in parallel and capture the data.

In this sample pointless code, I would like to get the value from the function but it is not assigned. If I can do this then I can call a function and pass it values in parallel.

#!/bin/bash
set -x

return_var=''

function pass_back_a_string() {
    eval "$1='foo bar rab oof'"
    sleep 2
}

pass_back_a_string return_var &
wait
echo $return_var

Answer 1

This can be done with bash 4.x and coprocesses:

#!/bin/bash
# REQUIRES BASH 4.1+

pbas() {
  sleep 10
  printf '%s\0' "hello, world: $RANDOM"
}

echo "Starting coprocesses $SECONDS seconds after interpreter startup"

# Start three coprocesses
coproc cp1 { pbas; }
coproc cp2 { pbas; }
coproc cp3 { pbas; }

# Read results from each in turn
IFS= read -r -d '' result1 <&${cp1[0]}
IFS= read -r -d '' result2 <&${cp2[0]}
IFS= read -r -d '' result3 <&${cp3[0]}

# Emit said results
echo "Result 1: $result1"
echo "Result 2: $result2"
echo "Result 3: $result3"

echo "Exiting $SECONDS seconds after interpreter startup"

You'll observe that it correctly exits roughly 10 seconds after interpreter startup, as opposed to the 30 you would get if each instance of the function ran in the parent shell.

---

Now, if you want to get a bit trickier (and, for this example, rely on bash 4.3), you can do even more interesting things -- reducing the amount of repetitive code:

#!/bin/bash
# REQUIRES BASH 4.3+

pbas() {
  sleep 10
  printf '%s\0' "hello, world: $RANDOM"
}

declare -A all_coprocs=( )
start_coproc() {
  local retval_name=$1; shift
  local eval_header eval_str eval_footer
  printf -v eval_header 'coproc cp_%q { ' "$retval_name"
  printf -v eval_str '%q ' "$@"
  printf -v eval_footer '; }'
  eval "${eval_header}${eval_str}${eval_footer}"
  all_coprocs[$retval_name]="cp_${retval_name}"
}

collect_results() {
  local retval_name
  for retval_name in "${!all_coprocs[@]}"; do
    declare -n cp=${all_coprocs[$retval_name]}
    IFS= read -r -d '' "$retval_name" <&${cp[0]}
    unset -n cp
  done
}

start_coproc result1 pbas
start_coproc result2 pbas
start_coproc result3 pbas
collect_results

echo "Result 1: $result1"
echo "Result 2: $result2"
echo "Result 3: $result3"

echo "Exiting $SECONDS seconds after interpreter startup"

---

Now, if you only have bash 3.x, then things get trickier: You don't have automatic FD allocation, or coprocesses, or support for redirecting from file descriptors named by variables [except by using eval]. However, you can generate named pipes; spawn off subprocesses that write to them; and then read from them in the collection phase.

#!/bin/bash
# REQUIRES BASH 3.2+

declare -a destvars=( )
declare -a destdirs=( )
declare -a destfds=( )
nextfd=10

pbas() { sleep 10; printf '%s\0' "hello, world: $RANDOM"; }

start_proc() {
  local destvar=$1; shift
  local destdir=$(mktemp -d "${TEMPDIR:-/tmp}/coproc-emulation.XXXXXX")
  local idx=$(( ${#destdirs[@]} + 1 ))
  local eval_str
  mkfifo "$destdir/outpipe"
  ( "$@" ) >"$destdir/outpipe" & procs[$idx]=$!
  printf -v eval_str 'exec %q<%q' "$nextfd" "$destdir/outpipe"
  eval "$eval_str"
  destdirs[$idx]=$destdir
  destvars[$idx]=$destvar
  destfds[$idx]=$((nextfd++))
}

collect_results() {
  local idx destdir destvar destfd
  for idx in "${!destvars[@]}"; do
    destvar=${destvars[$idx]}
    destdir=${destdirs[$idx]}
    destfd=${destfds[$idx]}
    printf -v eval_str 'IFS= read -r -d "" %q <&%q' "$destvar" "$destfd"
    eval "$eval_str"
    rm -rf "$destdir"
    unset destvars[$idx] destdirs[$idx] destfds[$idx]
  done
}

start_proc result1 pbas
start_proc result2 pbas
start_proc result3 pbas

collect_results

echo "Result 1: $result1"
echo "Result 2: $result2"
echo "Result 3: $result3"

echo "Exiting $SECONDS seconds after interpreter startup"