How to merge contours in opencv?

Question

Ok guys I have been working on this project for quite some time now.

I am building this bot that plays the chrome dinosaur game. So I tried other methods to detect the characters like matchTemplate and even made my own algorithm to locate the objects, but I like this one (findcontours) the most.

Here's what I have:

Can anyone help me find out how I should merge the two rectangles of the cacti?

img = screen_cap()
roi = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(roi,127, 255, 0)
im2, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
first = True
for cnt in contours:
    area = cv2.contourArea(cnt)
    if area > 200: #filtering contours
        x,y,w,h = cv2.boundingRect(cnt)
        if w/h < 4: # filtering even more
            cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)

Answer 1

Sorry to be a little late to the party. However if I google "merge opencv contours" I find this; and I think there should be an answer.

You can merge any two contours by one of those recipes:

• Get a list of points of each contour
• append them
• force them into cv2 contour format
• get cv2.convexHull of that if you don't care too much about the details.

If you don't like the result of convexHull because the concave parts of the contours are important, then follow this recipe instead:

• Get a list of points of each contour
• append them
• get a common center
• sort all the points in clockwise order around the center
• force them into cv2 contour format

If the two contours have a lot of concave shapes in them, this could yield a zigzag-pattern as the recipe goes through both contours disregarding their original structure. If that's the case, you need to follow a third recipe:

• Get a list of points of each contour
• get a common center
• delete points of each contour which are inside the other contour
• find the point which is closest to the common center in each contour.
• go through the first contour as it is listed until you hit the closest point.
• Then switch to the other list, starting from the closest point you go clockwise through the other contour until it is used up
• switch back to the first contour and append the rest of their points.
• force them into cv2 contour format

The next-more-complicated case is, if you have multiple intersections between the contours and you want to preserve the holes between the two. Then it's better to make a black image and draw the contours in white via cv2.fillPoly(); then get the contours back out via cv2.findContours()

I sketched some steps for the first two recipes here

get a list of points of each contour:

import cv2
list_of_pts = [] 
for ctr in ctrs_to_merge:
    list_of_pts += [pt[0] for pt in ctr]

order points clockwise

I use the function of this really great posting of MSeifert to order the points in clockwise order.

class clockwise_angle_and_distance():
    '''
    A class to tell if point is clockwise from origin or not.
    This helps if one wants to use sorted() on a list of points.

    Parameters
    ----------
    point : ndarray or list, like [x, y]. The point "to where" we g0
    self.origin : ndarray or list, like [x, y]. The center around which we go
    refvec : ndarray or list, like [x, y]. The direction of reference

    use: 
        instantiate with an origin, then call the instance during sort
    reference: 
    https://stackoverflow.com/questions/41855695/sorting-list-of-two-dimensional-coordinates-by-clockwise-angle-using-python

    Returns
    -------
    angle
    
    distance
    

    '''
    def __init__(self, origin):
        self.origin = origin

    def __call__(self, point, refvec = [0, 1]):
        if self.origin is None:
            raise NameError("clockwise sorting needs an origin. Please set origin.")
        # Vector between point and the origin: v = p - o
        vector = [point[0]-self.origin[0], point[1]-self.origin[1]]
        # Length of vector: ||v||
        lenvector = np.linalg.norm(vector[0] - vector[1])
        # If length is zero there is no angle
        if lenvector == 0:
            return -pi, 0
        # Normalize vector: v/||v||
        normalized = [vector[0]/lenvector, vector[1]/lenvector]
        dotprod  = normalized[0]*refvec[0] + normalized[1]*refvec[1] # x1*x2 + y1*y2
        diffprod = refvec[1]*normalized[0] - refvec[0]*normalized[1] # x1*y2 - y1*x2
        angle = atan2(diffprod, dotprod)
        # Negative angles represent counter-clockwise angles so we need to 
        # subtract them from 2*pi (360 degrees)
        if angle < 0:
            return 2*pi+angle, lenvector
        # I return first the angle because that's the primary sorting criterium
        # but if two vectors have the same angle then the shorter distance 
        # should come first.
        return angle, lenvector

center_pt = np.array(list_of_pts).mean(axis = 0) # get origin
clock_ang_dist = clockwise_angle_and_distance(origin) # set origin
list_of_pts = sorted(list_of_pts, key=clock_ang_dist) # use to sort

force a list of points into cv2 format

import numpy as np
ctr = np.array(list_of_pts).reshape((-1,1,2)).astype(np.int32)

merge them with cv2.convexHull instead

If you use this then there is no need to order the points clockwise. However, the convexHull might lose some contour properties because it does not preserve concave corners of your contour.

# get a list of points
# force the list of points into cv2 format and then
ctr = cv2.convexHull(ctr) # done.

I think the function to merge two contours should be a content of the opencv library. The recipe is quite straightforward and it is sad that many programmers, using opencv, will have to boilerplate code this.

Answer 2

The easiest way to merge contours is to stack them

contours = np.vstack(contours)

in your case(for example to stack contours 5 and 6):

contours = np.vstack([contours[5], contours[6]])

Answer 3

This is an old question and seems like it has not been answered properly yet (apologies to fellow SOers who did it partially in comments though). To me it seems like the questioner has a two-part question:

1. Is there a opencv function that merges two rectangles?

Answer to this question is yes and no. Let me be clear; yes if you are using opencv C++ bindings. Simple & can be used to take a union and | for intersection of two rects. But Python bindings lack those functions.

2. How to then do it in Python?

   def union(a,b):
       x = min(a[0], b[0])
       y = min(a[1], b[1])
       w = max(a[0]+a[2], b[0]+b[2]) - x
       h = max(a[1]+a[3], b[1]+b[3]) - y
       return (x, y, w, h)
   
   def intersection(a,b):
       x = max(a[0], b[0])
       y = max(a[1], b[1])
       w = min(a[0]+a[2], b[0]+b[2]) - x
       h = min(a[1]+a[3], b[1]+b[3]) - y
       if w<0 or h<0: return () # or (0,0,0,0) ?
       return (x, y, w, h)
       # Please remember a and b are rects.
   

Source Code Credit: OpenCV union and intersaction on rects