CODEWITHSUNDEEP
stringbashfor-loopawksplit
willeM_ Van Onsem
willeM_ Van Onsem

Reputation: 477180

Split string and pass individual string to a for loop in bash

Say you have a (multiline) string here named $cnt:

Foo
X
Y
Z
Foo
A
B
C
D

One can string the regex into multiple plarts with awk as follows:

awk '/Foo/{i++}{print > "dat"i}' <<<"$cnt"

The result is a large number of files dat1, dat2,...

But the different parts of the string must be processed by additional commands, so one can of course define a for loop:

for f in "dat*"
do
    #commands to process $f file
done

Files are however not a good solution for intermediate results since they are slow, global (they can interact with other files), use more memory (since all files are generated first whereas an iterative approach can reuse the memory released after processing the previous slice),... Therefore it would be better if one could process the instances as variables. Is there a convenient way to do this.

Note that the different parts can contain new lines thus that it is not sufficient to read the lines.

The first slice to process is thus:

Foo
X
Y
Z

The second slice should be:

Foo
A
B
C
D

Upvotes: 1

Views: 79

Answers (1)

Etan Reisner
Etan Reisner

Reputation: 80992

You can pipe directly to a process from awk instead of going through intermediate files if your process can handle the streamed input and you don't need to do anything fancy with the output from the process.

awk '/Foo/{i++; if (cmd){close(cmd); cmd="whatever command you need"}{print | cmd}' <<<"$cnt"

You could split the input and use \0 to delimit the segments and pipe to xargs -0 -n 1 whatever command, etc.

You could use a shell read loop for this by accumulating lines manually and calling the process on each new header.

while IFS= read -r line; do
    if [ "$line" = "$sentinel" ]; then
        if [ -n "$acc" ]; then
            whatever command "$acc"
        fi
        acc=""
    fi
    acc+="$line"$'\n'
done <<<"$cnt"

Upvotes: 1

Related Questions