CODEWITHSUNDEEP
bashparallel-processinggnu-parallel
Tim
Tim

Reputation: 7780

Splitting a string by space to pipe into parallel in bash

I have a piece of bash code that loops over filenames that are in a string separated by spaces. This then runs a function against each filename. It looks like so:

TCURR=0
for F in $FILES; do
  TCURR=$[TCURR + 1]
  do_something $TCURR $F &
done
wait

How can I make this work with gnu parallel? I can't figure out how to pipe the $FILES variable into it so it works.

Here's what I've tried, neither of which work. I based these off the parallel examples page. I'm just sending 1 instead of TCURR as the first variable to the function for simplicity although I would like to know how to do that too.

$FILES | parallel "do_something 1 {}"

echo $FILES | parallel "do_something 1 {}"

Upvotes: 1

Views: 1046

Answers (3)

Ole Tange
Ole Tange

Reputation: 33685

parallel do_something {#} {} ::: $FILES

Upvotes: 2

Mark Setchell
Mark Setchell

Reputation: 207435

You probably want this:

#!/bin/bash
FILES=....
TCURR=0
for F in $FILES; do
    ((TCURR++))
    echo do_something $TCURR $F 
done | parallel

Upvotes: 1

Tim
Tim

Reputation: 7780

Figured it out, just needed quotes around the variable. So it looks like:

echo "$FILES" | parallel "do_something 1 {}"

Although I haven't figured out how to make the counter work yet

Upvotes: 1

Related Questions