CODEWITHSUNDEEP
bash
Sander
Sander

Reputation: 1401

Running a dynamic amount of asynchronous tasks in bash

I have a bash-script that should run a synchronize task oil r cron:sync $id on all IDs in a config file, as I'm quite new to bash, I was wondering about how to do it.

The thing is, syncs have to be done every 10 seconds, and it should not wait for a sync, thus I'm using & between the syncs

Currently I have this file:

#!/bin/bash

cd /var/www/script/

i="0";

function dsync {
        oil r cron:sync $1
}


while [ $i -lt 10 ]; do
        echo ">> Syncing IDs - loop $i"
        dsync 126776804 & dsync 108792704 & dsync 108291602

        sleep 10
        i=$[$i+1]
done

Now the question is, what if I have a file with on each line the IDs that have to be synced, how would I go by doing this ?

Upvotes: 2

Views: 316

Answers (3)

clt60
clt60

Reputation: 63902

For variable count of parallel evocations of the dsync you can use the xargs in form:

xargs -n1 -I% -P0 oil r cron:sync % < filename_with_IDs

example with a pipe

seq 100010 100015 | xargs -n1 -I% -P0 echo oil r cron:sync %

produces

oil r cron:sync 100011
oil r cron:sync 100010
oil r cron:sync 100012
oil r cron:sync 100013
oil r cron:sync 100014
oil r cron:sync 100015

(ofc, you need remove the echo)

The -P0 ensures than any number of arguments will be executed in parallel (forked into background)

from the man:

-P max-procs

Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time. Use the -n option with -P; otherwise chances are that only one exec will be done.

Upvotes: 1

Mathias
Mathias

Reputation: 1500

I think what you want is something like:

#!/bin/bash

cd /var/www/script/

i="0";

function dsync {
        oil r cron:sync $1
}


while [ $i -lt 10 ]; do
        echo ">> Syncing IDs - loop $i"

        for j in $(cat ids.txt); do
          dsync $j &
        done

        sleep 10
        i=$[$i+1]
done

Upvotes: 1

hek2mgl
hek2mgl

Reputation: 157967

Your syntax when starting the tasks is wrong. To start them all in background you need to place them on separate lines:

dsync 126776804 &
dsync 108792704 &
dsync 108291602 &

You are further saying it should not wait for a sync. Having this, I assume that you don't care about the return status of the tasks, is this assumption correct?

Upvotes: 1

Related Questions