CODEWITHSUNDEEP
pythonopencvimage-processing
ravi pandit
ravi pandit

Reputation: 117

Finding coordinates of point closed by rectangle

Let say I have this image

image

The datapoint already detected and closed by the rectangle shape, as you can see in the figure. Now, I would like to find the coordinates of these rectangles and label them based on datapoint colours and saved them in CSV file.

For example, if the 1st image is in blue then I would like output something like this in CSV file

top, bottom, width, length, colour
   a      b     c        d    Blue

I want these values for each data point marked by the rectangle shape so that I can get the location of each data points. U can expect to have may data points for the given image, this just an example.

Any idea of how to do this? thanks in advance.

Upvotes: 0

Views: 215

Answers (1)

Prefect
Prefect

Reputation: 1777

Here is an approach:

img_original = cv2.cvtColor(cv2.imread('rectangles.jpg',1),cv2.COLOR_BGR2RGB)

img_rgb = img_original.copy()


img_rgb_0 = img_rgb[0:300,:]
img_rgb_1 = img_rgb[300:700,:]
img_rgb_2 = img_rgb[700:,:]

with open('coordinates.csv', "w+") as myfile:
    myfile.write("x_left_most,y_top_most,width,height,brush_color\n")

c= 0
for img_rgb in [img_rgb_0,img_rgb_1,img_rgb_2]:#,img_rgb_2]:

    img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_RGB2GRAY)


    ret,img_binary = cv2.threshold(img_gray,70,1,cv2.THRESH_BINARY_INV)
        
    img_binary = cv2.dilate(img_binary,np.ones((3,3),np.uint8),iterations = 3)


    skel = skeletonize(img_binary).astype(np.uint8)
    skel = cv2.dilate(skel,np.ones((3,3),np.uint8),iterations = 1)
    skel = np.amax(skel)-skel
    
    labels,stats= cv2.connectedComponentsWithStats(skel,connectivity=8)[1:3]
    x_left_most = stats[1:, cv2.CC_STAT_LEFT][1]
    y_top_most = stats[1:, cv2.CC_STAT_TOP][1]
    width = stats[1:, cv2.CC_STAT_WIDTH][1]
    height = stats[1:, cv2.CC_STAT_HEIGHT][1]


    img_gray[img_binary==1] = 255

    
    rectangle_color = img_rgb[y_top_most:y_top_most+height,x_left_most:x_left_most+width]
    rectangle = img_gray[y_top_most:y_top_most+height,x_left_most:x_left_most+width]

    # Get binary brushed point
    brush = np.zeros_like(rectangle)
    ret,rectangle_binary = cv2.threshold(rectangle.astype(np.uint8),250,1,cv2.THRESH_BINARY_INV)
    labels,stats= cv2.connectedComponentsWithStats(rectangle_binary,connectivity=8)[1:3]
    largest_label = 1 + np.argmax(stats[1:, cv2.CC_STAT_AREA])
    brush[labels == largest_label] = 1

    # Get color of brush
    brush_3D = np.dstack([brush,brush,brush])
    mask = brush_3D==0
    masked =rectangle_color.copy()
    masked[mask] = 0


    r,g,b = 0,0,0


    r = np.mean(masked[:,:,0][brush==1])
    g = np.mean(masked[:,:,1][brush==1])
    b = np.mean(masked[:,:,2][brush==1])

    color_index = np.argmax([r,g,b])

    colors = ['red','green','blue']
    brush_color = colors[color_index]

    print(f"x_left_most: {x_left_most}\ny_top_most : {y_top_most}\nwidth : {width}\nheight: {height}\ncolor: {brush_color}")


    fig,ax=plt.subplots()
    ax.imshow(img_rgb,cmap='gray')
    ax.scatter(x=[x_left_most,x_left_most,x_left_most+width,x_left_most+width],y=[y_top_most,y_top_most+height,y_top_most,y_top_most+height],c='r')
    plt.savefig(f"img_{c}.png")

    with open('coordinates.csv', "a") as myfile:
        myfile.write("{},{},{},{},{}\n".format(x_left_most,y_top_most,width,height,brush_color))
    c += 1
    plt.show()

img_0

x_left_most: 80 y_top_most : 59 width : 256 height: 199 color: blue

img_1

x_left_most: 80 y_top_most : 128 width : 253 height: 197 color: blue

enter image description here

x_left_most: 81 y_top_most : 85 width : 259 height: 203 color: red

I tried a few approaches to extract the rectangle and the datapoint (brush) in the rectangle. I achieved a working one, where I use two separate connected components, one for getting the coordinates and one for getting the color. A short guideline for the code:

  1. Skeletonize the rectangle lines to split the image into two regions: rectangle area and outside
  2. Get the rectangle area pixels by using the connected components
  3. Binarize the rectangle area to get the brush pixels
  4. Mask the brush with the brush pixels and get the brush pixels by using the connected components
  5. Get argmax of the each channel (RGB) to determine the color

Upvotes: 1

Related Questions