CODEWITHSUNDEEP
shellunix
user1967473
user1967473

Reputation: 73

concatenate files with similar names using shell

I have very limited knowledge of shell scripting, for example if I have the following files in a folder

abcd_1_1.txt
abcd_1_2.txt
def_2_1.txt
def_2_2.txt

I want the output as abcd_1.txt, def_2.txt. For each pattern in the file names, concantenate the files and generate the 'pattern'.txt as an output

patterns list <-?
for i in patterns; do echo cat "$i"* > "$i".txt; done

I am not sure how to code this in a shell script, any help is appreciated.

Upvotes: 2

Views: 1305

Answers (2)

John1024
John1024

Reputation: 113824

for i in abcd_1 def_2
do
    cat "$i"*.txt > "$i".txt
done

The above will work in any POSIX shell, such as dash or bash.

If, for some reason, you want to maintain a list of patterns and then loop through them, then it is appropriate to use an array:

#!/bin/bash
patterns=(abcd_1 def_2)
for i in "${patterns[@]}"
do
    cat "$i"*.txt > "$i".txt
done

Arrays require an advanced shell such as bash.

Related Issue: File Order

Does it the order in which files are added to abcd_1 or def_2 matter to you? The * will result is lexical ordering. This can conflict with numeric ordering. For example:

$ echo def_2_*.txt
def_2_10.txt def_2_11.txt def_2_12.txt def_2_1.txt def_2_2.txt def_2_3.txt def_2_4.txt def_2_5.txt def_2_6.txt def_2_7.txt def_2_8.txt def_2_9.txt

Observe that def_2_12.txt appears in the list ahead of def_2_1.txt. Is this a problem? If so, we can explicitly force numeric ordering. One method to do this is bash's brace expansion:

$ echo def_2_{1..12}.txt 
def_2_1.txt def_2_2.txt def_2_3.txt def_2_4.txt def_2_5.txt def_2_6.txt def_2_7.txt def_2_8.txt def_2_9.txt def_2_10.txt def_2_11.txt def_2_12.txt

In the above, the files are numerically ordered.

Upvotes: 2

ooga
ooga

Reputation: 15501

Maybe something like this (assumes bash, and I didn't test it).

declare -A prefix
files=(*.txt)
for f in "${files[@]"; do
    prefix[${f%_*}]=
done
for key in "${!prefix[@]}"; do
    echo "${prefix[$key]}.txt"
done

Upvotes: 2

Related Questions