CODEWITHSUNDEEP
awk
OXXO
OXXO

Reputation: 724

Merge multiple files and split output to multiple files based in each column (post 2)

I have many csv files with exactly same format on rows and columns. In the example below I present only 2 files as input, but i have a lot files with same characteristics

The purpose is for each input file do:

Take value in row 1, 2 and 3.

example in first file

6174
15
3

Then, print first column from row 4 to 6.

Do same process for all input files and output a file with all information of all readed files.

When the process is done for all files and first column. Do the same of the rest columns

At the end the total files output created will be 4 files as there is 4 columns in each file.

Input1

Record Number 6174
Vibrator Identification 15
Start Time Error 3 us
1.6,19.5,,,
1.7,23.2,28.3,27.0
1.8,26.5,27.0,25.4

Input2

Record Number 6176
Vibrator Identification 17
Start Time Error 5 us
1.6,18.6,,,
1.5,23.5,19.7,19.2
1.3,26.8,19.2,18.5

Using the code below, I got the 4 output files as desired, although files 3-4, are not good as spected, because in the first lines there is empty values and my code does not work as supposed. Also I have an issue to get the good value in row 3 in each file.. I get us instead of a number.

output file1

6174,15,3,1.6,1.7,1.8
6176,17,5,1.6,1.5,1.3

output file2

6174,15,3,19.5,23.2,26.5
6176,17,5,18.6,23.5,26.8

output file3

6174,15,3,0,0,28.3,27.0
6176,17,5,0,0,19.7,19.2

output file4

6174,15,3,0,0,27.0,25.4
6176,17,5,0,0,19.2,18.5

code used

The code works almost fine, merge the csv files and output the 4 files requerides, but there is a problem for files 3-4, when there is empty values.

        for f in *.csv ; do

        awk -F, 'NR==1 {n=split($NF,f," ");print f[n]}' "$f" >> a-"$f"
        awk -F, 'NR==2 {n=split($NF,f," ");print f[n]}' "$f" >> a-"$f"
        awk -F, 'NR==3 {n=split($NF,f," ");print f[n]}' "$f" >> a-"$f"
        sed -i  's/\r$//' a-"$f"

        for i in seq $(1...4); do
        awk -F, 'NR>=4{f=1} f{print '"$""$i"'} f==6{exit}' "$f" > "a""$i"-"$f"

            cat a-"$f" a"$i""-""$f" >> t"$i" 

            sed -i  's/\r$//' t"$i" 

        done

                for i in seq $(1...4); do
                awk -v RS= -v OFS=',' -v ORS='\n' '{$1=$1}1' t"$i" > file"$i".csv

    done

done

rm -f ./a*  ./t*

Appreciate your help

Upvotes: 0

Views: 54

Answers (1)

Ed Morton
Ed Morton

Reputation: 203324

With GNU awk for ENDFILE and automatic handling of multiple open files and assuming your posted sample output showing file3 and file4 each having more fields than file1 and file2 is a mistake:

$ cat tst.awk
BEGIN { FS=OFS=","; numHdrFlds=3 }
FNR <= numHdrFlds {
    gsub(/[^0-9]/,"")
    hdr = (FNR==1 ? "" : hdr OFS) $0
    next
}
{
    for (i=1; i<=NF; i++) {
        data[i] = (FNR==(numHdrFlds+1) ? "" : data[i] OFS) ($i)+0
    }
}
ENDFILE {
    for ( fileNr=1; fileNr<=NF; fileNr++ ) {
        print hdr, data[fileNr] > ("outputFile" fileNr)
    }
}

.

$ awk -f tst.awk file1 file2

$ for i in outputFile*; do echo "$i"; cat "$i"; echo "---"; done
outputFile1
6174,15,3,1.6,1.7,1.8
6176,17,5,1.6,1.5,1.3
---
outputFile2
6174,15,3,19.5,23.2,26.5
6176,17,5,18.6,23.5,26.8
---
outputFile3
6174,15,3,0,28.3,27
6176,17,5,0,19.7,19.2
---
outputFile4
6174,15,3,0,27,25.4
6176,17,5,0,19.2,18.5
---

Upvotes: 1

Related Questions