bob
bob

Reputation: 95

minMaxLoc() of cv2.matchTemplate() result gives implausible position, how can I fix this?

I am trying to write a python script that searching for an image on the screen using cv2.matchTemplate fucntion, and then extracting the x and y coordinates of that image and move the mouse to that location. But for some reason when I am running the code the mouse moves to the end of the screen and I do not understand why. Can someone help me figer it out?

this is my code:

import numpy as np
import mss
import cv2
import time
import pyautogui

# the templates to search for by order
TEMPLATES_BY_ORDER = ['EmailField.png',
                      'PasswordField.png', 'CheckBox.png', 'SignInButton.png']


def main():
    time.sleep(5)
    currentImage = 0

    with mss.mss() as sct:
        # Get the size of the primary monitor
        monitor = {"top": 0, "left": 0, "width": pyautogui.size()[
            0], "height": pyautogui.size()[1]}

        while currentImage < 1:
            # the path for the current template image
            template_img_path = f"images/{TEMPLATES_BY_ORDER[currentImage]}"

            # reading both the screen recording frame and the template image to search for
            screen_frame = np.array(sct.grab(monitor))[:, :, :3]
            template_img = cv2.imread(template_img_path, cv2.IMREAD_COLOR)

            screen_frame_dementions = screen_frame.shape

            resized_img = cv2.resize(
                template_img, (screen_frame_dementions[1], screen_frame_dementions[0]))

            # setting the next image in line
            currentImage += 1

            # matching the template to the screen record
            result = cv2.matchTemplate(
                screen_frame, resized_img, cv2.TM_CCOEFF_NORMED)

            # getting the coordinates of the template in the recording
            __, __, __, max_loc = cv2.minMaxLoc(result)
            x, y = max_loc
            pyautogui.moveTo(x, y)
            if cv2.waitKey(25) & 0xFF == ord("q"):
                cv2.destroyAllWindows()
                break


if __name__ == "__main__":
    main()

This is the screen that I am letting the code record and search in. enter image description here And this is the test.png image that I am trying to search for enter image description here

Upvotes: 0

Views: 153

Answers (0)

Related Questions