CODEWITHSUNDEEP
pythonopencv
E Rind
E Rind

Reputation: 53

Detecting a broken shape using OpenCV

I am trying to detect the square inside 4 bars. The goal is to compare their centroids. For the most part, my method (described below) correctly identifies the majority of cases. In images where the focus was bad or the lighting was too low, the processed image has a broken or disconnected square whose contour is not detected using Canny. Here is the general method flow:

  1. Use template matching to find the 'bars' and create an ROI
  2. Use adaptiveThreshold with ADAPTIVE_THRESH_MEAN_C on ROI
  3. Isolate the square by setting the edges of the ROI to 255 (white)
  4. Use morphologyEx with MORPH_OPEN and a (8,8) MORPH_RECT as the structuring element
  5. Use an optimized Canny via the imutils.auto_canny method (written by Adrian Rosebrock at pyimagesearch.com) to find the square
  6. Return the centers of the 'bars' and 'square' and display their minAreaRect's on the original image

Here is an example of a typical success: thresholded ROI, 'bars' removed, morphology applied, original with detected shapes

Here is an example of a broken, undetectable 'square': thresholded ROI, 'bars' removed, morphology applied, original with detected shapes

Things I've tried:

At step 2: 

methods = ['cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY)[1]',
           'cv2.threshold(cv2.GaussianBlur(gray, (5, 5), 0), 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]',
           'cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 51, 2)',
           'cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 51, 2)']

Global thresholding couldn't account for the varying brightnesses going from image to image. ADAPTIVE_THRESH_MEAN_C has a greater success rate than ADAPTIVE_THRESH_GAUSSIAN_C. The '51' in both of those was arbitrary; with testing it seemed like higher numbers produced a cleaner shape.

At step 4:

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (8, 8))
roi = cv2.morphologyEx(roi, cv2.MORPH_OPEN, kernel)

For structuring elements, I've used RECT, ELLIPSE, and CROSS with various sizes. If they get too big, I end up creating shapes inside the 'square' that auto.canny prefers to detect and somehow can't find the outer 'square' anymore. Using RETR_EXTERNAL instead of RETR_LIST, RETR_CCOMP, or RETR_TREE does not fix this issue. For morphology operations, I've tried MORPH_OPEN and dilate/erode separately as a means to close the broken 'square' shape prior to contour detection.

At step 5:

auto_canny seems to work well. I've tried template matching the 'square' after all this processing, but it fails far too often.

I'm looking for a singular solution which captures both the current successes and some of the more "achievable" failures. I'm open to any wisdom you have to offer. New directions that I'm currently thinking about:

While I hope that there is a magical method out there that does exactly what I'm looking for, I have spend several units of time working on this and expect a similarly difficult answer. Thank you in advance for your help!

Upvotes: 5

Views: 2170

Answers (1)

Stephen Meschke
Stephen Meschke

Reputation: 2940

Try filling in the square before finding the center. I did this by iterating through the rows and connecting the left-most and right-most black pixel.

filling in square

# Iterate through each row in the image
for row in range(img.shape[0]):
    # Find the left-most and right-most pixel in the sqare
    start, stop = 0, 0
    for col in range(img.shape[1]):
        # Find the left-most
        if img[row,col] != 255 and start == 0: start = col, row
        # Find the right-most
        if img[row,col] != 255: stop = col, row
    # If there was a pixel in that row, connect them with a line
    if start != 0:
        cv2.line(img, start, stop, 0, 1)

Upvotes: 4

Related Questions