matchtemplate openCv2 python not match similar look

Question

I have an Image that change it random the color, so some time the OpenCV recognize the object and sometimes it isn’t:

My code:

grey_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
grey_temp = cv2.cvtColor(temp, cv2.COLOR_BGR2GRAY)

w, h = grey_temp.shape[::-1]

res = cv2.matchTemplate(grey_img, grey_temp, cv2.TM_CCOEFF_NORMED)
print(res)

threshold = 0.8;
loc = np.where(res >= threshold)
print(loc)

for pt in zip(*loc[::-1]):
    cv2.rectangle(img, pt, (pt[0] + w, pt[1] + h), (255,0,0), 3)

cv2.imshow('win', img)
cv2.waitKey()

Simlar code with emguCv:

Mat background = CvInvoke.Imread(file, 
Emgu.CV.CvEnum.ImreadModes.AnyColor);//Convert Image to ReducedGrayscale2
Mat find = CvInvoke.Imread(compered, Emgu.CV.CvEnum.ImreadModes.AnyColor);
Mat match = CvInvoke.Imread(compered, Emgu.CV.CvEnum.ImreadModes.AnyColor);
CvInvoke.MatchTemplate(background, find, match, TemplateMatchingType.CcoeffNormed);         
double[] minValues, maxValues;
Point[] minLocations, maxLocations;
match.MinMax(out minValues, out maxValues, out minLocations, out maxLocations);
            if (maxValues[0] > 0.7)
            {
                Console.WriteLine("found");
                Rectangle match1 = new Rectangle(maxLocations[0],);
                match.Draw(match1, new Bgr(Color.Red), 3);
            }

        CvInvoke.Imshow("Image1", background);
        CvInvoke.WaitKey(0);

(Btw changing the threshold not contribute)

Example of the recognition:

I want to know if u have some idea how can I mange to track the same pattern object in the photo without depending the color, gray scale, or his location (the object has some similar shared properties so in human eye it called be tracked easily).

I tried doing the same progress with emguCv for c sharp to check if there any different in the result, but no surprises, no difference. I would appreciate if u can suggest me how to mange to track the template in the main image. Thanks in Advance!

Answer 1

One color insensitive approach is to do template matching on binary edge extracted images. So this seems to work in Python/OpenCV.

• Read the image and template as grayscale
• Do Canny edge detection with L2gradient=True
• Do Normalized Cross Correlation with those edge images
• Get the location of the maximum correlation value
• Put the template white edges in red into the image edges

Image:

Template:

import cv2
import numpy as np

# read image
img = cv2.imread('blue_object.png')

# convert img to grayscale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# read template as grayscale
tmplt = cv2.imread('pink_template.png', cv2.IMREAD_GRAYSCALE)
hh, ww = tmplt.shape

# do canny edge detection on image and template
img_edges = cv2.Canny(img_gray,100,200,3,L2gradient=True)
tmplt_edges = cv2.Canny(tmplt,100,200,3,L2gradient=True)

# do template matching
corrimg = cv2.matchTemplate(img_edges,tmplt_edges,cv2.TM_CCOEFF_NORMED)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corrimg)
max_val_ncc = '{:.3f}'.format(max_val)
print("normalize_cross_correlation: " + max_val_ncc)
xx = max_loc[0]
yy = max_loc[1]
print('xmatchloc =',xx,'ymatch =',yy)

# offset the templt_edges image to place at match point in black image the size of the img_edges
templt_edges_match = np.full_like(img_edges, 0)
templt_edges_match[yy:yy+hh, xx:xx+ww] = tmplt_edges

# draw template edges in red onto img edges
result1 = img_edges.copy()
result1 = cv2.merge((result1, result1, result1))
result1[templt_edges_match==255] = (0,0,255)

# draw template edges in red onto img
result2 = img.copy()
result2[templt_edges_match==255] = (0,0,255)

cv2.imshow('image', img)
cv2.imshow('template', tmplt)
cv2.imshow('image edges', img_edges)
cv2.imshow('template edges', tmplt_edges)
cv2.imshow('result1', result1)
cv2.imshow('result2', result2)
cv2.waitKey(0)
cv2.destroyAllWindows()

# save results
cv2.imwrite('edge_template_match_on_image_edges.png', result1)
cv2.imwrite('edge_template_match_on_image.png', result2)

Result1:

Result2:

Match Location:

xmatchloc = 59 ymatch = 36