Parallelizing a ping sweep

Question

I'm attempting to sweep an IP block totaling about 65,000 addresses. We've been instructed to use specifically ICMP packets with bash and find a way to parallelize it. Here's what I've come up with:

#!/bin/bash
ping() {
  if ping -c 1 -W 5 131.212.$i.$j >/dev/null
  then
      ((++s))
      echo -n "*"
  else
      ((++f))
      echo -n "."
  fi
  ((++j))
  #if j has reached 255, set it to zero and increment i
  if [ $j -gt 255 ]; then
      j=0
      ((++i))
      echo "Pinging 131.212.$i.xx IP Block...\n"
  fi
}

s=0 #number of responses recieved
f=0 #number of failures recieved
i=0 #IP increment 1
j=0 #IP increment 2
curProcs=$(ps | wc -l)
maxProcs=$(getconf OPEN_MAX)
while [ $i -lt 256 ]; do
    curProcs=$(ps | wc -l)
    if [ $curProcs -lt $maxProcs ]; then
      ping &
    else
      sleep 10
    fi
done
echo "Found "$s" responses and "$f" timeouts."
echo /usr/bin/time -l
done

However, I've been running into the following error (on macOS):

redirection error: cannot duplicate fd: Too many open files

My understanding is I'm going over a resource limit, which I've attempted to rectify by only starting new ping processes if the existing processes count is less than the specified max, but this does not solve the issue.

Thank you for your time and suggestions.

EDIT: There are a lot of good suggestions below for doing this with preexisting tools. Since I was limited by academic requirements, I ended up splitting the ping loops into a different process for each 12.34.x.x blocks, which although ugly did the trick in under 5 minutes. This code has a lot of problems, but it might be a good starting point for someone in the future:

#!/bin/bash

#############################
#      Ping Subfunction     #
#############################
# blocks with more responses will complete first since worst-case scenerio
# is O(n) if no IPs generate a response
pingSubnet() {
  for ((j = 0 ; j <= 255 ; j++)); do
    # send a single ping with a timeout of 5 sec, piping output to the bitbucket
    if ping -c 1 -W 1 131.212."$i"."$j" >/dev/null
    then
        ((++s))
    else
        ((++f))
    fi
  done
  #echo "Recieved $s responses with $f timeouts in block $i..."
  # output number of success results to the pipe opened in at the start
  echo "$s" >"$pipe"
  exit 0
}

#############################
#   Variable Declaration    #
#############################
start=$(date +%s) #start of execution time
startMem=$(vm_stat | awk '/Pages free/ {print $3}' | awk 'BEGIN { FS = "\." }; {print ($1*0.004092)}' | sed 's/\..*$//');
startCPU=$(top -l 1 | grep "CPU usage" | awk '{print 100-$7;}' | sed 's/\..*$//')
s=0 #number of responses recieved
f=0 #number of failures recieved
i=0 #IP increment 1
j=0 #IP increment 2

#############################
#    Pipe Initialization    #
#############################
# create a pipe for child procs to write to
# child procs inherit runtime environment of parent proc, but cannot
# write back to it (like passing by value in C, but the whole env)
# hence, they need somewhere else to write back to that the parent
# proc can read back in
pipe=/tmp/pingpipe
trap 'rm -f $pipe' EXIT
if [[ ! -p $pipe ]]; then
    mkfifo $pipe
    exec 3<> $pipe
fi

#############################
#     IP Block Iteration    #
#############################
# adding an ampersand to the end forks the command to a separate, backgrounded
# child process. this allows for parellel computation but adds logistical
# challenges since children can't write the parent's variables
echo "Initiating scan processes..."
while [ $i -lt 256 ]; do
      #echo "Beginning 131.212.$i.x block scan..."
      #ping subnet asynchronously
      pingSubnet &
      ((++i))
done
echo "Waiting for scans to complete (this may take up to 5 minutes)..."
peakMem=$(vm_stat | awk '/Pages free/ {print $3}' | awk 'BEGIN { FS = "\." }; {print ($1*0.004092)}' | sed 's/\..*$//')
peakCPU=$(top -l 1 | grep "CPU usage" | awk '{print 100-$7;}' | sed 's/\..*$//')
wait
echo -e "done" >$pipe

#############################
#    Concat Pipe Outputs    #
#############################
# read each line from the pipe we created earlier, adding the number
# of successes up in a variable
success=0
echo "Tallying responses..."
while read -r line <$pipe; do
    if [[ "$line" == 'done' ]]; then
      break
    fi
    success=$((line+success))
done

#############################
#    Output Statistics      #
#############################
echo "Gathering Statistics..."
fail=$((65535-success))
#output program statistics
averageMem=$((peakMem-startMem))
averageCPU=$((peakCPU-startCPU))
end=$(date +%s) #end of execution time
runtime=$((end-start))
echo "Scan completed in $runtime seconds."
echo "Found $success active servers and $fail nonresponsive addresses with a timeout of 1."
echo "Estimated memory usage was $averageMem MB."
echo "Estimated CPU utilization was $averageCPU %"

Answer 1

This should give you some ideas with GNU Parallel

parallel --dry-run -j 64 -k ping 131.212.{1}.{2} ::: $(seq 1 3) ::: $(seq 11 13)

ping 131.212.1.11
ping 131.212.1.12
ping 131.212.1.13
ping 131.212.2.11
ping 131.212.2.12
ping 131.212.2.13
ping 131.212.3.11
ping 131.212.3.12
ping 131.212.3.13

• -j64 executes 64 pings in parallel at a time
• -dry-run means do nothing but show what it would do
• -k means keep the output in order - (just so you can understand it)

The ::: introduces the arguments and I have repeated them with different numbers (1 through 3, and then 11 through 13) so you can distinguish the two counters and see that all permutations and combinations are generated.

Answer 2

Of course it's not as optimal as you are trying to build above, but you could start the maximum allow number of processes on the background, wait for them to end and start the next batch, something like this (except I'm using sleep 1s):

for i in {1..20}             # iterate some
do 
    sleep 1 &                # start in the background
    if ! ((i % 5))           # after every 5th (using mod to detect)
    then 
        wait %1 %2 %3 %4 %5  # wait for all jobs to finish
    fi
done

Answer 3

Don't do that.

Use fping instead. It will probe far more efficiently than your program will.

$ brew install fping

will make it available, thanks to the magic of brew.