CODEWITHSUNDEEP
linuxbashsedxargs
galactica
galactica

Reputation: 1833

how to batch sed files and redirect results

I wonder if there is a one-liner for batch processing a set of files in one folder and redirect the results into a different folder. I tried something like this:

find input_dir/ -name "PATTERN" | xargs -I {} sed 's:foo:bar:g' > output_dir/{}

For example, input_dir/ has file A, B, C and my hoped result is to have processed files A, B, C in output_dir/, with the same file names.

my hope was to use {} to replace file names and to build the output file paths, but this didn't work.

Anyone knows how to fix this? Or other better ways of doing so? Thanks!

Upvotes: 1

Views: 1265

Answers (3)

F. Hauri - Give Up GitHub
F. Hauri - Give Up GitHub

Reputation: 70882

In two steps:

find input_dir/ -name "PATTERN" -exec cp -t output_dir/ {} +

than

sed 's:foo:bar:g' -i output_dir/*

or, if output_dir could contain files not matching "PATTERN":

find output_dir -name "PATTERN" -exec sed -e 's:foo:bar:g' -i {} +

Upvotes: 1

Ole Tange
Ole Tange

Reputation: 33725

Using GNU Parallel it looks like this:

find input_dir/ -name "PATTERN" | parallel sed s:foo:bar:g {} '>' output_dir/{/}

If the sed command has special chars, then you need to quote those double:

find input_dir/ -name "PATTERN" | parallel sed 's:foo.\*:bar:g' {} '>' output_dir/{/}

Upvotes: 1

Jonathan Leffler
Jonathan Leffler

Reputation: 754520

My technique for this is to write a shell script that does the job, and then run it via find. For example, your actions could be written into a script munger.sh:

#!/bin/sh
for file in "$@"
do
    output="output_dir/$(basename "$file")"
    sed -e 's:foo:bar:g' "$file" > "$output"
done

The find command becomes:

find input_dir -name "PATTERN" -exec sh munger.sh {} +

This runs the script with the file names as arguments, bundling conveniently large number of file names into a single invocation of the shell script. If you're not going to need it again, you can simply remove munger.sh when you're done.

Yes, you can do all sorts of contortions to execute the command the way you want (perhaps using find … -exec bash -c "the script to be executed" arg0 {} +) but it is often harder than writing a relatively simple script and using it and throwing it away. There tend to be fewer problems with quoting, for example, when you run an explicit script than when you try to write the script on the command line. If you find yourself fighting with single quotes, double quotes and backslashes (or back-quotes), then it is time to use a simple script as shown.

Upvotes: 3

Related Questions