CODEWITHSUNDEEP
shell
AnPocArBuile
AnPocArBuile

Reputation: 665

Shell Script: Using files based on part of file name

I have a directory (say ~/dir/) containing several hundred files. Half of these will begin with the string "ABC", ie they will be called ABC_0.csv, ABC_1.csv, ABC_2.csv etc etc.

My goal is to write a shell script that will take each of these "ABC" files and merge them together in one bigger file which I am calling "master_ABC".

I know how to merge them, but I do not know how to write a shell script that will only take files with names beginning in "ABC" (Note: There are other files in the ~/dir/ that I have no interest in and want to avoid).

Moreover, the number of "ABC" files will vary from day to day.

Upvotes: 0

Views: 398

Answers (4)

afenster
afenster

Reputation: 3608

Use ls (or find) with grep in while loop:

ls | grep 'ABC_.*\.csv$' | while read fn ; do cat $fn >> master_ABC.csv ; done

or with find (especially if you need to traverse subdirectories recursively):

find . -type f -name 'ABC*.csv' | while read fn ; do cat $fn >> master_ABC.csv ; done

Note that grep accepts a regex, while find required a wildcard string.

I'd suggest to avoid using * in such cases as it will not work for very long list of files, and will also fail if any of the file names contain a space character.

Upvotes: 1

Amar
Amar

Reputation: 3845

You can use wildcard * for this.

#!/bin/bash

cat ~/dir/ABC*csv > master_ABC

Upvotes: 1

Knight71
Knight71

Reputation: 2949

You could capture all the files in a list and then cat with append (>>) to master file

files=`ls ABC*csv`

 for f in $files
 do
   echo $f
   cat $f >> master_ABC.csv
 done

Upvotes: 1

nu11p01n73R
nu11p01n73R

Reputation: 26667

Use wildcard * expansion to get different files ABC_1.csv and so on

cat ABC_*.csv > master_ABC.csv

Upvotes: 2

Related Questions