CODEWITHSUNDEEP
bashawksedgrep
Nirbhay Singh
Nirbhay Singh

Reputation: 409

Redirect a process output in a csv/excel file in bash script

I have written a script in bash which generates output of disk I/O performance and gives output in following format which i redirect to a text file. 

2018-04-18-12-09-32
File Size 250KB
dir /opt/testfile
WRITE TEST ==> 116 MB/s
READ TEST (W/O CACHE) ==> 350 MB/s
READ TEST (WITH CACHE) ==> 657 MB/s

But i want it to generate this output in a .csv file with following table format extracting from text file.

Date-time              Filename (being tested)     Filesize     Test type                      Speed
2018-04-18-12-09-32    /opt/testfile               1MB          READ TEST (W/O CACHE)           350 MB/s
2018-04-18-12-09-32    /opt/testfile               1GB          Write TEST (W/O CACHE)          500 MB/s

I tried few things using awk, sed, cut but i am not able to generate the results.

Suggestions will be a great help. Thanks in advance guys :)

Upvotes: 0

Views: 896

Answers (2)

R.Liu
R.Liu

Reputation: 34

one liner with awk

echo "Date-time              Filename (being tested)     Filesize     Test type                      Speed";cat 1.txt | tr '\n' '|' | nawk -F"|" '/WRITE TEST/ {split($2,a," ");printf("%-22s%-29s%-13s%-30s%-10s\n",$1,substr($3,index($3," ")),a[3],substr($4,0,index($4,"=")-1),substr($4,index($4,">")+1))} /READ TEST \(W\/O CACHE\)/ {printf("%-22s%-29s%-13s%-30s%-10s\n",$1,substr($3,index($3," ")),a[3],substr($5,0,index($5,"=")-1),substr($5,index($5,">")+1))} /READ TEST \(WITH CACHE\)/ {printf("%-22s%-29s%-13s%-30s%-10s\n", $1,substr($3,index($3," ")),a[3],substr($6,0,index($6,"=")-1),substr($6,index($6,">")+1))}'
Date-time              Filename (being tested)     Filesize     Test type                      Speed
2018-04-18-12-09-32    /opt/testfile               250KB        WRITE TEST                     116 MB/s
2018-04-18-12-09-32    /opt/testfile               250KB        READ TEST (W/O CACHE)          350 MB/s
2018-04-18-12-09-32    /opt/testfile               250KB        READ TEST (WITH CACHE)         657 MB/s

Upvotes: 1

tshiono
tshiono

Reputation: 22042

Although I've not fully understood your input file format without the complete set of input file and expected output, I've tried to make a best guess. Assuming the input file name is "textfile", how about:

#!/bin/bash

declare -a datetimes
declare -A filesize
declare -A dir
declare -A testtype
declare -A speed

# extract parameters with regex from the input file
while read -r line; do
    if [[ "$line" =~ ^([0-9]+-[0-9]+-[0-9]+-[0-9]+-[0-9]+-[0-9]+)$ ]]; then
        datetime="${BASH_REMATCH[1]}"
        datetimes+=($datetime)
    elif [[ "$line" =~ ^File\ Size\ +([[:alnum:]]+)$ ]]; then
        filesize[$datetime]="${BASH_REMATCH[1]}"
    elif [[ "$line" =~ ^dir\ +([^[:blank:]]+)$ ]]; then
        dir[$datetime]="${BASH_REMATCH[1]}"
    elif [[ "$line" =~ ^(.*TEST.*)\ +==\>\ +([0-9]+.+)$ ]]; then
        test="${BASH_REMATCH[1]}"
        testtype[$datetime]+="${test},"
        speed[$datetime,$test]="${BASH_REMATCH[2]}"
    fi
done < textfile

# report the results in csv format
printf "%s,%s,%s,%s,%s\n" "Date-time" "Filename (being tested)" "Filesize" "Test type" "Speed"
for d in ${datetimes[@]}; do
    test="${testtype[$d]}"
    ifs_b="$IFS"
    IFS=,
    for t in $test; do
        printf "%s,%s,%s,%s,%s\n" "$d" "${dir[$d]}" "${filesize[$d]}" "$t" "${speed[$d,$t]}"
    done
    IFS="$ifs_b"
done

Sample of textfile modified from the original one:

2018-04-18-12-09-32
File Size 250KB
dir /opt/testfile
WRITE TEST ==> 116 MB/s
READ TEST (W/O CACHE) ==> 350 MB/s
READ TEST (WITH CACHE) ==> 657 MB/s
2018-04-19-01-23-45
File Size 1GB
dir /opt/testfile2
WRITE TEST ==> 120 MB/s
READ TEST (W/O CACHE) ==> 300 MB/s
READ TEST (WITH CACHE) ==> 600 MB/s

And the result will be:

Date-time,Filename (being tested),Filesize,Test type,Speed
2018-04-18-12-09-32,/opt/testfile,250KB,WRITE TEST,116 MB/s
2018-04-18-12-09-32,/opt/testfile,250KB,READ TEST (W/O CACHE),350 MB/s
2018-04-18-12-09-32,/opt/testfile,250KB,READ TEST (WITH CACHE),657 MB/s
2018-04-19-01-23-45,/opt/testfile2,1GB,WRITE TEST,120 MB/s
2018-04-19-01-23-45,/opt/testfile2,1GB,READ TEST (W/O CACHE),300 MB/s
2018-04-19-01-23-45,/opt/testfile2,1GB,READ TEST (WITH CACHE),600 MB/s

Hope this helps.

Upvotes: 1

Related Questions