OpenCV: Can't find large rectangle contour

Question

Opencv vers: 4.5

I'm trying to re-create the dimensions of an object by setting it up on a grid and taking as close to a top-down photo I can which I will then get the contours of the largest bounding rectangle and then perspective warp.

I'm currently unable to get the contour for a large bounding square however, it continually only finds smaller rectangles/squares which I'm assuming would not be large enough to properly fix the perspective.

First image: Original

Second image: What I get with my code using openCV

Third image: Close to what I'd ideally get

My code:

import imutils
import numpy as np
import cv2 as cv

# load the query image
image = cv.imread("path/to/image")

# make image greyscale, blur, find edges
grayscale_image = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
thresh = cv.adaptiveThreshold(grayscale_image, 255, cv.ADAPTIVE_THRESH_GAUSSIAN_C,
                              cv.THRESH_BINARY, 11, 2)

# find contours in the threshed image, keep only the largest
# ones
cnts = cv.findContours(
    thresh.copy(), cv.RETR_LIST, cv.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
cnts = sorted(cnts, key=cv.contourArea, reverse=True)[:5]

# draw contours for reference
cv.drawContours(image, cnts, -1, (0, 255, 0), 3)

Instead of adaptive thresholding for pre-processing I've tried using bilateral filter or gaussian blur into canny edge detection but the outcome still doesn't find large rectangles.

Any help would be greatly appreciated as I'm at a loss on why it can't detect larger squares. Also, if people think there's a better method for fixing the perspective so that I can accurately recreate the board dimensions please let me know.

Answer 1

You may apply the following stages:

• Apply threshold using cv2.threshold (instead of cv2.adaptiveThreshold).
• Apply opening with long column vector for keeping only the vertical lines.
• Find contours in vert_lines.
• Sort contours left to right.
• Draw most left and most right contours on a sketch (black) image.
• Apply opening with long row vector for keeping only the horizontal lines, find contours, sort top to bottom, and draw top and bottom contours.
• Find inner contours in the sketch image (with the left, right, top and bottom lines).
  The inner contour is the smallest one.

---

Here is a code sample:

import imutils
import numpy as np
import cv2

# load the query image
image = cv2.imread("image.png")

# make image greyscale, blur, find edges
grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

#thresh = cv2.adaptiveThreshold(grayscale_image, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)
thresh = cv2.threshold(grayscale_image, 0, 255, cv2.THRESH_OTSU)[1]  # Apply automatic threshold (use THRESH_OTSU).

rect_im = np.zeros_like(thresh)  # Sketch image

# Apply opening with long column vector for keeping only the vertical lines.
vert_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, np.ones(50))

# Apply opening with long row vector for keeping only the horizontal lines.
horz_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, np.ones((1,50)))

# Find contours in vert_lines
cnts = imutils.grab_contours(cv2.findContours(vert_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE))

# Sort contours left to right.
cnts = sorted(cnts, key=lambda c: cv2.boundingRect(c)[0])  # cv2.boundingRect(c)[0] is the left side x coordinate.

cv2.drawContours(rect_im, [cnts[0], cnts[-1]], -1, 255, -1) # Draw left and right contours

# Find contours in horz_lines
cnts = imutils.grab_contours(cv2.findContours(horz_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE))

# Sort contours top to bottom.
cnts = sorted(cnts, key=lambda c: cv2.boundingRect(c)[1])  # cv2.boundingRect(c)[1] is the top y coordinate.

cv2.drawContours(rect_im, [cnts[0], cnts[-1]], -1, 255, -1)  # Draw top and bottom contours

# Find contours in rect_im
cnts = imutils.grab_contours(cv2.findContours(rect_im, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE))  # Note: use RETR_TREE for getting inner contour.

c = min(cnts, key=cv2.contourArea)  # Get the smallest contour

# Draw contour for reference
cv2.drawContours(image, [c], -1, (0, 255, 0), 3)

---

Results:

thresh:

vert_lines:

horz_lines:

Left and right lines:

rect_im:

image (output):