CODEWITHSUNDEEP
regexbashshellsed
Shawn McBride
Shawn McBride

Reputation: 552

Comparing files, but ignoring specific differences, in a shell script

I have a shell script that compares files of the same name across two directories and alerts me to differences. Most of the time the files files in a new batch are unchanged but occasionally one changes and I need to take some action. I'm trying to update my script to ignore minor differences that I don't care about, specifically Expiration and Signature tags in Amazon S3 URL's.

My current script uses cmp to just compare the files exactly:

if ! cmp --silent ${i} ${COMPARE_DIR}/${i}; then
  echo "$i : TAKE ACTION"
fi

I worked out fairly quickly on the command line a way to regex out the offending pattern using sed:

cmp <(sed -e "s/Expires.*3D/Expires3D/" file1.json) <(sed -e "s/Expires.*3D/Expires3D/" ../other/file1.json)

For the life of me I can't figure out the syntax to make this work in my script. First it complained about the parentheses, and after escaping those I got an oh-so-helpful No such file or directory error. Nothing else I've tried improves things.

Any suggestions? Is there a better way to get at what I want?

Upvotes: 0

Views: 65

Answers (3)

Mad Physicist
Mad Physicist

Reputation: 114300

If your action can be written as a single command or function call, you can use the && and || operators to replace if-then and else for a really messy one-liner:

 cmp <(sed -e "s/Expires.*3D/Expires3D/" file1.json) <(sed -e "s/Expires.*3D/Expires3D/" ../other/file1.json) && echo "$i : TAKE ACTION" || echo "$i: No Action"

Upvotes: 0

janos
janos

Reputation: 124646

First it complained about the parentheses, and after escaping those I got an oh-so-helpful No such file or directory error.

Most probably your script is running as Bash. Make sure the first line is #!/bin/bash (or wherever you have Bash in your system). This is because the <(...) is Bash specific, and may not be supported in /bin/sh or whatever it's linked to.

For example, if the first line of your script is #!/bin/sh, then you will get an error like this:

script.sh: line 3: syntax error near unexpected token `('

Then, as you say, you "escaped the parentheses", like <\(sed -e ...\), the shell will interpret that as input redirection from a file, and since you don't have such file (a file named "(sed"), the No such file or directory error is exactly as expected.

Upvotes: 1

Mad Physicist
Mad Physicist

Reputation: 114300

Assuming that the line cmp <(sed -e "s/Expires.*3D/Expires3D/" file1.json) <(sed -e "s/Expires.*3D/Expires3D/" ../other/file1.json) works properly, you can check the result as follows:

cmp <(sed -e "s/Expires.*3D/Expires3D/" file1.json) <(sed -e "s/Expires.*3D/Expires3D/" ../other/file1.json)
if [ "$?" -ne "0" ] ; then
    echo "$i : TAKE ACTION"
fi

Upvotes: 1

Related Questions