CODEWITHSUNDEEP
shell
user121196
user121196

Reputation: 31020

append csv files one by one in numeric order using bash shell script

Given files named 1.csv, 2.csv, 3.csv, ...89.csv... n.csv how do I append them together in numeric order)(1 to n) in bash shell script? is there a one-liner for the solution?

Upvotes: 4

Views: 1486

Answers (2)

servn
servn

Reputation: 36

One liner with inline params (you should replace N with maximal file number). Script will skip unexisted files, so if you miss few files in sequence you wouldn't get error :)

for i in `seq 0 N`; do test -f $i.csv && cat $i.csv; done >> result.csv

Or script with params

#!/bin/bash

for i in `seq 0 $1`
do
    if [ -f "$i.csv" ]
    then
        cat $i.csv >> $2
    fi
done

Usage: ./script.sh MAX_CSV_ID RESULT_FILE

Example: ./script.sh 89 new.csv

Upvotes: 0

shellter
shellter

Reputation: 37288

If your files where named with leading zeros, it would be easier, i.e.

  cat [0-9].csv [0-9][0-9].csv .... > new.csv

But its not too hard to get a true numeric order, given 

ls -1
1
10
11
12
13
2
20
21
3
7
8
9

(in both samples, note that the option to ls is the number one, (1), not the letter L (l))

AND

ls -1 [0-9]* | sort -n
1
2
3
7
8
9
10
11
12
13
20
21

THEN 

  cat $( ls -1 *.csv | sort -n  ) > new.csv

Assuming all your csv files are numbered.

If you have more than 1000 files, file arg processing in the shell may break down, and you should post a new question for correct use of xargs.

To see what is happening add shell debugging/trace use

 set -vx  # to turn on
 set +vx  # to turn it off

.

IHTH.

Upvotes: 2

Related Questions