CODEWITHSUNDEEP
bashshellcsvwhile-loop
GURU SINGH
GURU SINGH

Reputation: 149

Iterating through Comma Separated rows in loop in Shell

Final.txt

Failed,2021-12-07 22:30 EST,Scheduled Backup,abc,/clients/FORD_1030PM_EST_Windows2008,Windows File System
Failed,2021-12-07 22:00 EST,Scheduled Backup,def,/clients/FORD_10PM_EST_Windows2008,Windows File System

I want to iterate through these rows instead of column

Expected Output

client=abc
client=def
group=/clients/FORD_1030PM_EST_Windows2008
group=/clients/FORD_10PM_EST_Windows2008

I tried this

while read line ; do 
    group=$(awk -F',' '{print $4}')
    client=$(awk -F',' '{print $5}')
    echo $group
    echo $client
done < Final

it's Not working but when I am individually doing this

cat Final | awk -F',' '{print $4}'

then it is giving me the expected output but does not work when I am trying in the loop.

Upvotes: 1

Views: 787

Answers (4)

ufopilot
ufopilot

Reputation: 3985

LOG="Failed,2021-12-07 22:30 EST,Scheduled Backup,abc,/clients/FORD_1030PM_EST_Windows2008,Windows File System
Failed,2021-12-07 22:00 EST,Scheduled Backup,def,/clients/FORD_10PM_EST_Windows2008,Windows File System"

getColumnTextSed(){
        log="$1"
        column=$2
        [ -n "$log" -a -n "$column" ]  && sed -E 's/([^,]*),([^,]*),([^,]*),([^,]*),([^,]*),(.*)/\'$column'/mg;t;d' <<<$log
}
getColumnTextAWK(){
        log="$1"
        column=$2
        [ -n "$log" -a -n "$column" ]  && awk -F',' -v "c=$column" '{print $(c)}' <<<$log
}



echo "# with sed and column number"
echo "-------------------------------"
getColumnTextSed "$LOG" 4
getColumnTextSed "$LOG" 5
echo "-------------------------------"

echo "# with awk and column number"
echo "-------------------------------"
getColumnTextAWK "$LOG" 4
getColumnTextAWK "$LOG" 5

OUTPUT
# with sed
-------------------------------
abc
def
/clients/FORD_1030PM_EST_Windows2008
/clients/FORD_10PM_EST_Windows2008
-------------------------------
# with awk
-------------------------------
abc
def
/clients/FORD_1030PM_EST_Windows2008
/clients/FORD_10PM_EST_Windows2008

Upvotes: 0

glenn jackman
glenn jackman

Reputation: 247012

with bash

declare -a client group
while IFS=, read -ra fields; do 
    client+=("${fields[3]}")
    group+=("${fields[4]}")
done < Final
printf 'client=%s\n' "${client[@]}"
printf 'group=%s\n' "${group[@]}"

Using , it's no different really from a single-pass awk solution, collecting the values in arrays:

mlr --icsv --implicit-csv-header put '
    @client[NR] = $4;
    @group[NR] = $5;
    filter false;
    end {
        emit @client, "client";
        emit @group, "group";
    }
' Final

The equivalent of the above, as more readable (IMO) awk code:

awk -F, '
    {client[NR] = $4; group[NR] = $5}
    END {
        for (i=1; i<=NR; i++) print "client=" client[i]
        for (i=1; i<=NR; i++) print "group=" group[i]
    }
' Final

Using csvtool is nice because it has a transpose function, but it still needs help getting to the desired output

csvtool col 4,5 Final \
| csvtool cat <(echo "client,group") - \
| csvtool transpose - \
| awk -F, -v OFS="=" '{for (i=2; i<=NF; i++) print $1, $i}'

Upvotes: 0

Cyrus
Cyrus

Reputation: 88776

With GNU awk:

awk -F ',' 'BEGINFILE{f++}
            f==1{print "client=" $4}
            f==2{print "group=" $5}
           ' Final Final

Output:

client=abc
client=def
grooup=/clients/FORD_1030PM_EST_Windows2008
grooup=/clients/FORD_10PM_EST_Windows2008

Upvotes: 2

markp-fuso
markp-fuso

Reputation: 35046

One-pass awk solution, storing field $5 in an array for printing at the end:

$ awk -F, '{print $4; groups[NR]=$5} END {for (i=1;i<=NR;i++) print groups[i]}' Final.txt
abc
def
/clients/FORD_1030PM_EST_Windows2008
/clients/FORD_10PM_EST_Windows2008

Two-pass awk that eliminates need to store field $5 in an array:

$ awk -F, 'FNR==NR {print $4;next} {print $5}' Final.txt Final.txt
abc
def
/clients/FORD_1030PM_EST_Windows2008
/clients/FORD_10PM_EST_Windows2008

Upvotes: 0

Related Questions