CODEWITHSUNDEEP
pythonperformanceopencvchesstemplate-matching
j15
j15

Reputation: 21

Increase performance of template matching in OpenCV

I'm currently writing a project in python with OpenCV that looks to identify the position of the pieces on a chess board on screen.

I've successfully been able to use the template matching function to identify the location of each piece. This is done by getting a screenshot of the entire board, and running the template matching function 12 times (6 types of pieces for each colour). Now I am looking to improve the speed of the program. Here is where I believe I can drastically increase performance:

As far as I understand the template matching algorithm basically slides the template image over the base image one pixel at a time to look for matches. Obviously I would only expect matches in the 64 squares of the board, so moving the image one pixel at a time is incredibly inefficient. Is there some way of customizing the sliding interval to match the width of one square of the board, so I only check the 64 squares. Are there any other suggestions for how to improve speed?

I had considered the possibility of calling the template matching function 64 times for each type of pieces, to match with each individual square of the same size. I'm not sure if this would even be any faster, plus getting 64 screen shots would take a considerable amount of time as well.

Upvotes: 1

Views: 5296

Answers (2)

Yves Daoust
Yves Daoust

Reputation: 181

  • You don't have to slide across the squares as the pieces must be approximately centered.

  • You can probably perform the matching on a "generic" piece just to find the location; when you have the (approximate) location, you can try every piece, in a smaller window.

  • You can perform a fist search on a reduced image (and reduced templates) to get an initial approximation; then refine to full resolution.

Upvotes: 1

Hugo
Hugo

Reputation: 157

I faced the same challenge.

FASTER SOLUTION - Template Matching on same size subtraction

As you suggested, "calling the template matching function 64 times for each type of pieces, to match with each individual square of the same size" is a lot faster. But you don't need to get 64 screenshots. Only 1 screenshot will do, and then you get 64 subtractions.

full_image= cv2.imread('path_to_screenshot')
square_subtraction = full_image[yi:yf,xi:xf]

If the chess board is not always on the same position on the screen, you should first search for it on the full_screen and then create subtractions.

FASTEST SOLUTION - Pixel Signature

It is also possible that each piece has a unique pixel for some specific x,y coordinate that would make it different from all other 11 pieces. I call that "pixel signature".

If you can find this signature, you can use pixel matching instead of template matching from the 1st solution.

That's my code to find unique pixels over a set of templates.

from os import walk

def compare_pixel_RGB(rgb_to_compare, row, column):
    n = 0         
    for item in templates:
        rgb = templates[item][row][column] 
        if (rgb_to_compare == rgb).all() :
            n += 1
    return n

#Load template images. They should all have the same size.
templates = dict()
for (dirpath, dirnames, filenames) in walk('templates_board/'):
    for filename in filenames:
        if filename.endswith('.png'):
            template = cv2.imread('templates_board/'+filename)
            templates.update({filename: template})

#Create and populate matching DataFrame with same dimensions of the templates. 
#Each cell of the DataFrame corresponds to a pixel of the templates and has the sum of matching pixels for all pieces on this specific coordinate.
#We are hoping to find a cell = 12, meaning that each piece only matches itself and not any other one.
matching_list = list()
random_item = list(templates.keys())[0]
matching = pd.DataFrame(np.zeros((templates[random_item].shape[0],templates[random_item].shape[1])))                          
for item in templates:
    for row in range(templates[item].shape[0]):
        for column in range(templates[item].shape[1]):
            rgb = templates[item][row][column] 
            count_same_pixel = compare_pixel_RGB(rgb, row, column)
            matching_list.append([item, row, column, count_same_pixel])
            matching[column][row] = matching[column][row] + count_same_pixel

If you find the pixel signature coordinate (column x row from above), you can use it to get pixel subtractions for each of the 64 square of the board and compare against your templates pixel signatures:

full_image = cv2.imread('path_to_screenshot')
square_subtraction = full_image[yi:yf,xi:xf]
square_pixel = square_subtraction[column][row]

template = cv2.imread('path_to_template')
template_signature = template[column][row]

Just itinerate over each square of the board and compare with each template, exaclty like the 1st solution, but instead of template matching, you just need to compare the pixel.

template_signature  == square_pixel

Upvotes: 1

Related Questions