Template matching - cross correlation vs square diff - when choose one over the other?

Question

TL;DR

When and why is cross correlation beneficial over square diff (when using template matching)?

Details

According to OpenCV's documentation of template matching (you have to scroll down a bit), square difference is defined as:

and cross correlation as:

(where T is the template and I the image)

If I'm not mistaken, square diff is the only method (also compared to cross correlation coefficient) that is guaranteed to find the best match (in a numerical sense) in each image.

If we take a look at cross correlation we notice that the multiplication yields higher results for brighter parts in the image (because bright pixels have a higher numerical value than dark pixels). This means, if we perform template matching with a dark template on a bright image we will most likely get a bad result when using cross correlation.

For example, if we take this image:

and perfrom template matching with this template:

we get these results (red is cross correlation and green is square diff):

Obviously, square diff is the better choice here as cross correlation yields a very bad result. The only advantage I can see in using cross correlation over square diff is the computational complexity as cross correlation should be a bit faster because it only has to calculate a product (instead of a sum and a square).

My question is: are there any benefits that I don't see? When should I choose one method over the other?

Answer 1

Rosa Gronchi is right, you should use normalized cross-correlation. The Normalized Cross Correlation measurement is the Cross Correlation of the normalized vectors so that all vectors have length 1 and mean 0.
This way brighter patches won't have "advantage" over darker patches.

Regarding complexity, the square diff also have to calculate the product because it is extended to:

The product is calculated using the Fast Fourier Transform algorithm.

Answer 2

Square distance and correlation are both statistics coeffincets used in order to determine similarity.

The main difference between them lies in data type they are used to describe. Relation between both coefficents is well described here:

https://stats.stackexchange.com/a/34066

In short: Difference between both factors rely on standard deviation of probe.

In case you provided, result of correlation matching makes me wonder if you used pixel values in ranges (0,255) or (0,1). While processing values in range (0,255), classical correlation could match your object with more or less brightest region of image.

If you have any additional question, please leave a comment.

Have fun coding!