CODEWITHSUNDEEP
bashcsvunixmerge
yatici
yatici

Reputation: 577

merge csv files in unix, bash

I have a few csv files that are formatted like:

test1.csv:

    field,port1
    a1,0.2
    a2,0.3
    a3,0.6

test2.csv:

field,port2
b1,0.5
b2,0.6
b3,0.7
b4,0.1
b5,0.5

test3.csv:

field, port3
c1,0.1
c2,0.4

and so forth.I want to merge these csvs into a single so it would look like:

field,port1,field,port2,field,port3
a1,0.2,b1,0.5,c1,0.1
a2,0.3,b2,0.6,c2,0.4
a3,0.6,b3,0.7,,
,,b4,0.1,,
,,b5,0.5,,

how can I do this? I cat >> but that would but everything in the first two columns. I can go about that way if I have to but having a merge like this can make my life a lot simpler.

Thanks

Upvotes: 3

Views: 3549

Answers (2)

Walter Tross
Walter Tross

Reputation: 12624

Building on fedorqui's answer:

paste -d: test[1-3].csv | sed -e's/^:/,:/' -e's/::/:,:/g' -e's/::/:,:/g' -e's/:$/:,/' -e's/:/,/g'

(assuming you have no : in your files - but you can choose another temporary separator)

This restores all the commas you expect. The pair of identical substitution instructions is needed because a substituted string is not taken into account for another substitution.

In general:

paste -d'T' file... | sed -e's/^T/ET/' -e's/TT/TET/g' -e's/TT/TET/g' -e's/T$/TE/' -e's/T/S/g'

where T is the temporary separator (: above), E is the string that should replace an empty or missing line (, above), and S is the separator between the lines of the pasted files (, above). The temporary separator T (a generic string) must not appear in the files and in E, while the final separator S can.

Warning: The above commands may need spaces before the quoted strings in your shell

Upvotes: 1

fedorqui
fedorqui

Reputation: 289725

paste can do quite a similar thing:

$ paste -d, test[1-3].csv
field,port1,field,port2,field, port3
a1,0.2,b1,0.5,c1,0.1
a2,0.3,b2,0.6,c2,0.4
a3,0.6,b3,0.7,
,b4,0.1,
,b5,0.5,

Note that -d, stands for delimiter to be a comma.

Upvotes: 4

Related Questions