OpenCV: How to extract the inner area of a series of rectangles that are arranged in a particular way?

Question

I have a series of images which are like the image A. My intention is to detect the inner area of each rectangle and extract that area for further processing; so what I need are the position (x,y) and dimension (width,height) of the the areas which are shown with green in image B.

My current approach (which works just fine) is that I have created a PNG overlay of the areas that I need which have a specific color (image C), put the PNG overlay on top of the source image (with the hardcoded x,y), Performing a basic thresholding operation on the overlaid image using OpenCV to create a mask, finding all the contours, getting the position (x,y) and dimension (width,height) of all the matches and finally cropping those areas on the original image.

#defining the mask color
magenta_color = np.uint8([[[255, 0, 255]]])
hsv_magenta = cv2.cvtColor(magenta_color,cv2.COLOR_BGR2HSV)
lower_magenta = np.array(hsv_magenta[0][0])
upper_magenta = np.array(hsv_magenta[0][0])

#loading images
image_source = cv2.imread("source_image.png")
image_overlay = cv2.imread("overlay_image.png")

#putting overlay on top of original image at the correct position
image_overlaid = add_image(image_source, image_overlay, 20, 35)


#thresholding and finding the matches
image_overlaid = cv2.cvtColor(image_overlaid, cv2.COLOR_BGR2HSV)
masked = cv2.inRange(image_overlaid, lower_magenta, upper_magenta)
all_contours, _ = cv2.findContours(masked, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

#getting the x, y, width, height of each match
for contour in all_contours:
    x, y, width, height = cv2.boundingRect(contour)
    
    #cropping the area on the original image
    area = image_source[y:y + height, x:x + width]

As I said, the approach that I use works but it is very fragile because I should know (beforehand) the exact (x,y) position of overlay image and both source image and overlay image should match perfectly (in terms of position of rectangles and their dimensions).

I also tried to look for all the rectangles in the image (again using cv2.findContours) and use the found contours for getting the needed areas but as image A is evident, the black color has bled over many rectangles so I cannot check if a contour is actually a rectangle.

image_source = cv2.imread("source_image.png")
image_gray = cv2.cvtColor(image_source, cv2.COLOR_BGR2GRAY)
image_binary_inverse = cv2.adaptiveThreshold(image_gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 255, 10)
image_contours, _ = cv2.findContours(image_binary_inverse, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)

for contour in image_contours:
    cv2.drawContours(image_source, [contour], -1, (0, 255, 0), 2)

Is there a better approach for achieving what I intend that doesn't have the drawbacks of what I currently use (PNG overlay)?

Note: I have control over the generation process of the layout of the rectangles (or whatever that can be added to the layout before being filled)