CODEWITHSUNDEEP
pythonopencv
IronWaffleMan
IronWaffleMan

Reputation: 2712

How to complete/close a contour in python opencv?

I have a Pi camera pointed at a card on a white background. However, local shadows seem to be preventing the closing of the contours that I use for card detection, which means detection fails overall. Here's a screenshot of what I mean:

Screenshot of open contours

You can see it gets ragged around the bottom corners in particular. This is the code I'm using to get this far:

gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
gray = cv2.blur(gray, (5,5))
gray = cv2.bilateralFilter(gray, 11, 17, 17) #blur. very CPU intensive.
cv2.imshow("Gray map", gray)

edges = cv2.Canny(gray, 30, 120)

cv2.imshow("Edge map", edges)

#find contours in the edged image, keep only the largest
# ones, and initialize our screen contour
# use RETR_EXTERNAL since we know the largest (external) contour will be the card edge.
_, cnts, _ = cv2.findContours(edges.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = sorted(cnts, key = cv2.contourArea, reverse = True)[:1]
screenCnt = None

# loop over our contours
for c in cnts:
    # approximate the contour
    peri = cv2.arcLength(c, True)
    approx = cv2.approxPolyDP(c, 0.3 * peri, True)

    cv2.drawContours(image, [cnts[0]], -1, (0, 255, 0), 2)

    # if our approximated contour has four points, then
    # we can assume that we have found our card
    if len(approx) == 4:
        screenCnt = approx;
    break

Is there a way to force it to close specific contours? If I blur the image more to smooth the shadows that doesn't work either since it simply ignores those corners as not having an edge. It's annoying that it's merely a few pixels away from closing the contours, yet it never does...

edit: I now have a more realistic setup where the background is a beige colour and with a lot more shadows interfering. Beige is necessary because there are some cards with white borders, so white wouldn't work. The edge detection fails mostly in the left side where the shadows are.

enter image description here

Upvotes: 24

Views: 41054

Answers (2)

Denys Parchenko
Denys Parchenko

Reputation: 685

I recently encountered the same issue due to changing light conditions, my contour would be broken in multiple places to the extent that would not allow to close the gaps with morphological transformations:

Original image Dilated image

I was looking for a rectangular contour of the LCD display, which as you can see is broken in 2 places. Depending on the lighting conditions those gaps could appear in different locations. I ended up finding a convex hull of every contour which would always be closed and then approximating it down to a 4-vertex polygon:

def get_rectangular_contours(contours):
    """Approximates provided contours and returns only those which have 4 vertices"""
    res = []
    for contour in contours:
        hull = cv2.convexHull(contour)
        peri = cv2.arcLength(hull, closed=True)
        approx = cv2.approxPolyDP(hull, 0.04 * peri, closed=True)
        if len(approx) == 4:
            res.append(approx)
    return res

Then, in the list of those 4-vertex polygons I'd look for the one that resembles a rectangle with specific aspect ratio. It works like a charm:

Found contours Found rectangle

You can check out the full algorithm here.

Upvotes: 2

Eliezer Bernart
Eliezer Bernart

Reputation: 2426

As I mentioned in my comment to you answer, one of the easiest ways to "connect" the lines in the border is using morphological operators. In the following code, the edges of the image are dilated using an ellipsoid shape. This technique allows us to merge the lines that are close and fill some of the empty spaces. You can have more information about this topic in the OpenCV Documentation.

kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(9,9))
dilated = cv2.dilate(image, kernel)
_, cnts, _ = cv2.findContours(dilated.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

Here you can see the original edge image, the dilated image and the contour obtained using the dilated edges (image obtained using a cropped region of your original screenshot):

But, as you can see and also imagine, solving this issue for more general cases is more complex and will demand the usage of other approaches, and probably is broader than a SO question (or at least in the way it is formulated now).

By looking at your more difficult case, I could recommend you using other image representations to replace the grayscale input image (such as the H channel from HSV colorspace) in order reduce or attenuate the effects you are having with shadows. You could also explore some of the constraints in your problem: cards always have straight lines as borders and use a method capable of dealing with parametric forms, such as Hough Lines detector. Have a look at this question, it may give you some insights about how to improve your results: How to identify square or rectangle with variable lengths and width by using javacv?

Remark: Bilateral filtering is very computationally expensive, especially if you are using an RPi to run your application. I would recommend investing in some other alternatives, such as a Gaussian filtering, to reduce the amount of noise in the picture (assuming that you really need to do that).

Upvotes: 28

Related Questions