CODEWITHSUNDEEP
pythonopencvcomputer-visiontemplate-matching
Luiz
Luiz

Reputation: 140

How to reduce false positive on openCV template match on a software GUI

I'm trying to create a tasker to use on a game, like a bot to do repetitive simple tasks.

Some of the elements I'm looking for (like texts and buttons) are very similar (see images below).

I'm even using masks to acquire the best results, and the size of the templates is exactly the same as the original picture.

So, on the main screen of the game, only one of the elements must match, so I can decide what action the bot must perform, but these are the grades of each matchmaking:

img/btnCara.png 0.960

img/btnCoroa.png 0.960

img/btnFinalizar.png 0.988

img/btnRender.png 0.875

img/btnSim.png 0.997

img/btnJogar.png 0.922 << the only one on the image

img/txtEnemyChosen.png 0.953

img/txtJogarPrimeiro.png 0.945

img/txtVocePerdeu.png 0.951

So, how can I reduce the false positives and find only what I'm looking for on the screen?

The code I'm using is here:

def SearchImage(img):
    partial_image = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    partial_image= cv2.threshold(partial_image, 0, 255, cv2.THRESH_BINARY)[1]

    # get largest contour from binary image
    contours = cv2.findContours(partial_image.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    contours = contours[0] if len(contours) == 2 else contours[1]
    big_contour = max(contours, key=cv2.contourArea)

    # draw the contour of the piece outline as the mask
    mask = np.zeros((img.shape[0], img.shape[1], 3), dtype=np.uint8)
    cv2.drawContours(mask, [big_contour], 0, (255,255,255), 1)
    hh, ww = mask.shape[:2]

    # extract the template from the BGR (no alpha) piece 
    template = img[:,:,0:3]
    correlation = cv2.matchTemplate(img_np, template, cv2.TM_CCORR_NORMED, mask=mask)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(correlation)
    max_val_ncc = '{:.3f}'.format(max_val)
    #print("normalize_cross_correlation: " + max_val_ncc)
    xx = max_loc[0]
    yy = max_loc[1]
    #print(xx, yy)
        
    # draw template bounds and corner intersection in red onto img
    cv2.rectangle(screenshot, (xx, yy), (xx+ww, yy+hh), (0, 0, 255), 1)
    
    #return xx, yy
    return max_val_ncc

This is the GUI of the game:

GUI of the game

and this is the list of elements I'm looking for on the GUI

enter image description here

As you can see, some of them are very similar, like these two:

enter image description hereenter image description here

Upvotes: -1

Views: 500

Answers (2)

Shades of Dark
Shades of Dark

Reputation: 1

try threshold = cv2.adaptiveThreshold(edges,255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 201, 4) it creates a lot better contrast for text and might also help.

Upvotes: 0

Luiz
Luiz

Reputation: 140

Looks like the GUI has some elements that are very similar and that caused the openCV to mismatch some of them.

Using Canny Edge Detector technique (OpenCV Documentation about it) I Could remove unwanted information from the picture and achieved best results.

Upvotes: 0

Related Questions