OpenCV Template Matching for backgammon checker positions

Question

I am trying to analyze backgammon board and output number of checkers on each position for both players in this format:

    "o": {
      "6": 5,
      "8": 3,
      "13": 5,
      "24": 2
    },
    "x": {
      "1": 2,
      "12": 5,
      "17": 3,
      "19": 5
    }
  }

Board Image:

Checker Image:

Output:

Code:

import numpy as np
from matplotlib import pyplot as plt
img_rgb = cv.imread('./board.png')
assert img_rgb is not None, "file could not be read, check with os.path.exists()"
img_gray = cv.cvtColor(img_rgb, cv.COLOR_BGR2GRAY)
template = cv.imread('./black.png', cv.IMREAD_GRAYSCALE)
assert template is not None, "file could not be read, check with os.path.exists()"
w, h = template.shape[::-1]
res = cv.matchTemplate(img_gray,template,cv.TM_CCOEFF_NORMED)
threshold = 0.6
loc = np.where( res >= threshold)
for pt in zip(*loc[::-1]):
    print(pt[0] + w, pt[1] + h)
    cv.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 0)
cv.imwrite('res.png',img_rgb)

As you see issue is that not all checkers get matched and also so far its not quite clear how to count number of checkers on each position. Any advice would be helpful. Thanks