CODEWITHSUNDEEP
regexstringbashcsvparsing
angel_30
angel_30

Reputation: 1

Parse the output of my bash script and save as CSV

I have a bash script that SSH'es to a list of servers (given a .txt file), runs another script inside each server, and shows the results. But I need to parse the verbose data from output, and eventually save some meaningful results as a .CSV file.

Here is my main script:

set +e
while read line
do
    ssh myUser@"$line" -t 'sudo su /path/to/script.sh' < /dev/null
done < "/home/listOfServers.txt"

where the listOfServers.txt is like

server1
server2
server3

The output of running my script looks like this, showing the results for each servers one after another.

     SNAME:WORKFLOW_APS_001     |10891    | Alive:2018-06-18:06:54          |TCP
     SNAME:WORKFLOW_APSWEB_001     |11343    | Alive:2018-06-18:06:54          |TCP
    Processes in Instance: WORKFLOW_OHS_002
    WORKFLOW_OHS_002                 | OHS                |    8925 | Alive    |  852960621 |  1367120 | 510:11:51 | http:9881
    Processes in Instance: WORKFLOW_OHS_003
    WORKFLOW_OHS_003                 | OHS                |    9187 | Alive    | 2041606684 |  1367120 | 510:11:51 | http:9883
     SNAME:WORKFLOW_RPSF_001     |10431    | Alive:2018-06-18:06:55          |TCP
     SNAME:WORKFLOW_SCPTL_001     |9788    | Alive:2018-06-18:06:55          |TCP
...

From this output, I only need the OHS names and their status, and save along with the original server's name as a CSV. The pattern to me looks like this: I need to look at each line, and if the line doesn't contain "Processes in Instance" or "SNAME", then split based on space, and grab the 1st (OHS name) and 4th field (status). So my CSV will look like:

server1, WORKFLOW_OHS_002, Alive
server1, WORKFLOW_OHS_003, Alive
server2, .....
...

How can I modify my bash to do this?

Upvotes: 1

Views: 435

Answers (2)

anubhava
anubhava

Reputation: 785581

You can use awk:

while read -r line; do
    ssh myUser@"$line" -t 'sudo su /path/to/script.sh' < /dev/null |
    awk -v s="$line" -F '|' -v OFS=', ' '!/^[[:blank:]]*SNAME:/ && NF>2 {
       gsub(/^[[:blank:]]+|[[:blank:]]+$/, "");
       gsub(/[[:blank:]]*\|[[:blank:]]*/, "|");
       print s, $1, $4
    }'
done < "/home/listOfServers.txt"

EDIT: As per your comment below, you can do this to handle error conditions:

while read -r line; do
    out=$(ssh myUser@"$line" -t 'sudo su /path/to/script.sh' < /dev/null 2>&1)
    if [[ -z $out ]]; then
       echo "$line, NULL, NULL"
    elif [[ $out == *"timed out"* ]]; then
       echo "$line, FAIL, FAIL"
    else
       awk -v s="$line" -F '|' -v OFS=', ' '!/^[[:blank:]]*SNAME:/ && NF>2 {
          gsub(/^[[:blank:]]+|[[:blank:]]+$/, "");
          gsub(/[[:blank:]]*\|[[:blank:]]*/, "|");
          print s, $1, $4
       }' <<< "$out"
    fi
done < "/home/listOfServers.txt"

Upvotes: 1

Paul Hodges
Paul Hodges

Reputation: 15388

Something to try - and good luck.

while read line
do  ssh myUser@"$line" -t 'sudo su /path/to/script.sh' < /dev/null |
        sed -E "/Processes in Instance/d; /SNAME/d;
        s/^ *([^| ]*) *[|][^|]*[|][^|]*[|] *([^| ]*).*/$line,\1,\2/;" 
done < "/home/listOfServers.txt"

You ought to be able to improve on that. :)

Upvotes: 1

Related Questions