How do I "diff" multiple files against a single base file?

Question

I have a configuration file that I consider to be my "base" configuration. I'd like to compare up to 10 other configuration files against that single base file. I'm looking for a report where each file is compared against the base file.

I've been looking at diff and sdiff, but they don't completely offer what I am looking for.

I've considered diff'ing the base against each file individually, but my problem then become merging those into a report. Ideally, if the same line is missing in all 10 config files (when compared to the base config), I'd like that reported in an easy to visualize manner.

Notice that some rows are missing in several of the config files (when compared individually to the base). I'd like to be able to put those on the same line (as above).

Note, the screenshot above is simply a mockup, and not an actual application.

I've looked at using some Delphi controls for this and writing my own (I have Delphi 2007), but if there is a program that already does this, I'd prefer it.

The Delphi controls I've looked at are TDiff, and the TrmDiff* components included in rmcontrols.

Answer 1

But none of the solutions does more than 3 files still. What I did was messier, but for the same purpose (comparing contents of multiple config files, no limit except memory and BASH variables)

While loop to read a file into an array:

loadsauce () {
index=0
while read SRCCNT[$index]
 do let index=index+1
 done < $SRC
}

Again for the target file

loadtarget () {
index=0
while read TRGCNT[$index]
 do let index=index+1
 done < $TRG
}

string comparison

brutediff () {
# Brute force string compare, probably duplicates diff
# This is very ugly but it will compare every line in SRC against every line in TRG
# Grep might to better, version included for completeness
for selement in $(seq 0 $((${#SRCCNT[@]} - 1)))
 do for telement in $(seq 0 $((${#TRGCNT[@]} - 1)))
  do [[ "$selement" == "$telement" ]] && echo "${selement} is in ${SRC} and ${TRG}" >> $OUTMATCH
  done
 done
}

and finally a loop to do it against a list of files

for sauces in $(cat $SRCLIST)
 do echo "Checking ${sauces}..."
  loadsauce
  loadtarget
  brutediff
  echo -n "Done, "
 done

It's still untested/buggy and incomplete (like sorting out duplicates or compiling a list for each line with common files,) but it's definitely a move in the direction OP was asking for. I do think Perl would be better for this though.

Answer 2

For people that are still wondering how to do this, diffuse is the closest answer, it does N-way merge by way of displaying all files and doing three way merge among neighboors.

Answer 3

I know this is an old thread but vimdiff does (almost) exactly what you're looking for with the added advantage of being able to edit the files right from the diff perspective.

Answer 4

None of the existing diff/merge tools will do what you want. Based on your sample screenshot you're looking for an algorithm that performs alignments over multiple files and gives appropriate weights based on line similarity.

The first issue is weighting the alignment based on line similarity. Most popular alignment algorithms, including the one used by GNU diff, TDiff, and TrmDiff, do an alignment based on line hashes, and just check whether the lines match exactly or not. You can pre-process the lines to remove whitespace or change everything to lower-case, but that's it. Add, remove, or change a letter and the alignment things the entire line is different. Any alignment of different lines at that point is purely accidental.

Beyond Compare does take line similarity into account, but it really only works for 2-way comparisons. Compare It! also has some sort of similarity algorithm, but it also limited to 2-way comparisons. It can slow down the comparison dramatically, and I'm not aware of any other component or program, commercial or open source, that even tries.

The other issue is that you also want a multi-file comparison. That means either running the 2-way diff algorithm a bunch of times and stitching the results together or finding an algorithm that does multiple alignments at once.

Stitching will be difficult: your sample shows that the original file can have missing lines, so you'd need to compare every file to every other file to get the a bunch of alignments, and then you'd need to work out the best way to match those alignments up. A naive stitching algorithm is pretty easy to do, but it will get messed up by trivial matches (blank lines for example).

There are research papers that cover aligning multiple sequences at once, but they're usually focused on DNA comparisons, you'd definitely have to code it up yourself. Wikipedia covers a lot of the basics, then you'd probably need to switch to Google Scholar.

• Sequence alignment
• Multiple sequence alignment
• Gap penalty

Answer 5

I made my own diff tool DirDiff because I didn't want parts that match two times on screen, and differing parts above eachother for easy comparison. You could use it in directory-mode on a directory with an equal number of copies of the base file. It doesn't render exports of diff's, but I'll list it as a feature request.

Answer 6

SourceGear Diffmerge is nice (and free) for windows based file diffing.

Answer 7

Diff3 should help. If you're on Windows, you can use it from Cygwin or from diffutils.

Answer 8

for f in file1 file2 file3 file4 file5; do echo "$f\n\n">> outF; diff $f baseFile >> outF; echo "\n\n">> outF; done

Answer 9

Try Scooter Software's Beyond Compare. It supports 3-way merge and is written in Delphi / Kylix for multi-platform support. I've used it pretty extensively (even over a VPN) and it's performed well.

Answer 10

You might want to look at some Merge components as what you describe is exactly what Merge tools do between the common base, version control file and local file. Except that you want more than 2 files (+ base)...
Just my $0.02