CODEWITHSUNDEEP
linuxbashshellunix
Annie Jeba
Annie Jeba

Reputation: 379

Merge multiple files in multiple directories - bash

We have a requirement to loop multiple directiories and in each directory there will be, multiple text files with Pattern File"n".txt which needs to merged to one, File.txt

We are using Bourne Shell scripting.

Example:

/staging/dusk/inbound/ --> Main Directory

Dir1
File1.txt,File2.txt,..sample1.doc,sample2.pdf,File*.txt  --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Dir2
File1.txt,File2.txt,..attach1.txt,sample1.doc, File*.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Dir3
File1.txt,File2.txt,File*.txt,..sample1,sample2*.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Dir4
File1.txt,File2.txt,File*.txt,..temp.doc,attach.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Di5
File1.txt,File2.txt,File*.txt,..sample1,sample2*.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

Dir"n"
File1.txt,File2.txt,File3.txt,File*.txt..attach1,attach*.txt --> we have to Merge the File name which starts with Fil*.txt -->Final.txt

The files from each directory can be looped using cat *.txt > all.txt command.

But how we loop the directiories?

Upvotes: 1

Views: 5051

Answers (3)

abhishek phukan
abhishek phukan

Reputation: 791

you can probably try using the find command

find /staging/dusk/inbound/ -name "*.txt" -type f -exec cat {} \;>>MergedFile.txt

Upvotes: 1

Kernelv5
Kernelv5

Reputation: 1772

You can do it in a ad-hok way

find /staging/dusk/inbound -type f -name *.txt | sort -n | awk '{ print "cat " $1 " >> all.txt"}' > marge.sh
chmod +x marge.sh
sh merge.sh

Upvotes: 0

tripleee
tripleee

Reputation: 189397

To catenate Fil*.txt from all immediate subdirectories of the current directory,

cat */Fil*.txt >Final.txt

To access arbitrarily deeply nested subdirectories, some shells offer ** as an extension, but this is not POSIX sh -compatible.

cat **/Fil*.txt >Final.txt

If you actually want to loop over your directories and do something more complex with each, that's

for d in */; do
    : something with "$d"
done

Or similarly, for shells which support **, you can loop over all directories within directories;

for d in **/; do
    : something with "$d"
done

For example, : something with "$d" could be cat "$d"/Fil*.txt >"$d"/Final.txt to create a Final.txt in each directory, which contains only the Fil*.txt files in that directory.

Upvotes: 2

Related Questions