CODEWITHSUNDEEP
pythonopencvtemplate-matching
Devkil39
Devkil39

Reputation: 81

Match Template With Transparent Threshold

has anyone used template matching on OpenCV to detect images using transparent templates and it worked?

I'm trying to use template matching to detect multiple boxes that make up a table. The template used is a box of the same size but with a transparent background because the contents of the cell can be in the form of text with any font size.

Here the Image

Here the Template 1

Here the Template 2

img = cv.imread('image.png',-1)
template1 = cv.imread('template 1.png',-1)
template2 = cv.imread('template 2.png',-1)

h,w,_  = template.shape
method = cv.TM_CCOEFF_NORMED
res = cv.matchTemplate(img,template1,method)
# res = cv.matchTemplate(img,template2,method)

loc = np.where(res >= 0.8)

boxes = list()
for pt in zip(*loc[::-1]):
    boxes.append((pt[0],pt[1],pt[0]+w,pt[1]+h))

boxes = non_max_suppression(np.array(boxes))

for (x1, y1, x2, y2) in boxes:
    cv.rectangle(tem, (x1, y1), (x2, y2),(23,255,255), 1)

When running the match template module using template1, the results of the process fail to detect the existing boxes, whereas when using template2, the results of the process are able to detect boxes but only 1 out of 4 boxes can be detected.

The 4 boxes can be detected if the threshold used is lowered to 0.5, but when tested using other images it fails to detect the boxes correctly (sometimes the detection results are more than they should be, and sometimes the coordinates don't match).

From the result, I think the result appears because the match template process for transparent image failed to run and the frequency of appearance of black color in each box still affects the method.

If we look at the tutorial it shows that using a mask can solve this problem, but I am confused about which mask to use. In this example, the mask used is defined and there is no mention of how to create the mask. Anyone can help?

Upvotes: 1

Views: 605

Answers (1)

Devkil39
Devkil39

Reputation: 81

Following @fmw42 example in the link, I have managed to detect the box as needed.

However, in its implementation, adjustments need to be made because the use of black color as a template (border) followed by an transparent background which is also black causes the detection process to fail.

Here I solve it by changing the black color in template (border) to red color so that the template is composed of a red color in the border and a transparent black color. The colors used may be adjusted to your liking.

img = cv.imread('image.png')
img[:,:,:2] = 255 # change colors to red

template_all = cv.imread('template 1.png',cv.IMREAD_UNCHANGED)
template = template_all[:,:,0:3]
template[:,:,2] = 255 # change colors to red

alpha = template_all[:,:,3]
alpha = cv.merge([alpha,alpha,alpha])
 
h,w  = template.shape[:2]

method = cv.TM_CCORR_NORMED
res = cv.matchTemplate(img,template,method,mask=alpha)

loc = np.where(res >= 0.9)
result = img.copy()

boxes = list()
for pt in zip(*loc[::-1]):
    boxes.append((pt[0],pt[1],pt[0]+w,pt[1]+h))

boxes = non_max_suppression(np.array(boxes))

for (x1, y1, x2, y2) in boxes:
    cv.rectangle(result, (x1, y1), (x2, y2),(23,255,255), 1)

cv.imshow('result',result)
cv.waitKey(0)
cv.destroyAllWindows()

Upvotes: 1

Related Questions