CODEWITHSUNDEEP
unixheaderfile-copying
arun
arun

Reputation: 11023

How to efficiently append contents of a header file to a lot of files?

I have about 400 plain-text files of size roughly 350MB each. I want to prepend the contents of a header file (which contains 7 lines of plain text) to each one of these 400 files.

Currently I am looping over the 400 files and doing it with cat, followed by mv. Here is the pseudo-code:

for $infile in $indir {
    cat $headerfile $infile > $infile.tmp
    mv $infile.tmp $infile
}

Is there a more efficient way to do this?

Upvotes: 0

Views: 47

Answers (1)

Lekensteyn
Lekensteyn

Reputation: 66465

Insertion in a file is not possible without copying the remaining part of a file. Your pseudo-code is the most "efficient" method in terms of operations.

What might help is writing the intermediate file to tmpfs to avoid writing twice to disk:

cat "$headerfile" "$infile" > /tmp/tmp
mv /tmp/tmp "$infile"

(change tmp as needed if you want to run parallel commands).

Upvotes: 2

Related Questions