How to rotate bounding box in open CV and crop it (python)

Question

I have the coordinates of a rectangle (XMIN,YMIN,XMAX & YMAX) on a particular image. I wish to rotate the rectangle at a particular angle and then crop it from the image. How do I do that??

For example this image.i have got the output bounding box appearing on the left side (plotted it using XMIN,YMIN,XMAX & YMAX). I want to rotate it as per the image on the right side and then crop it.

Can someone provide the way to get this output with a sample code or point me to a link with the explanation

Answer 1

You can use a rotation matrix to rotate both the images and the bounding boxes.

Steps:

1. Generate a rotation matrix
2. Use OpenCV warpAffine to rotate the image
3. Rotate the 4 corners of the bounding box using the same rotation matrix

Read about getRotationMatrix2D and warpAffine

    img_path = '' #The image path
    bb = [] #XMIN,YMIN,XMAX,YMAX
    angle = 3 #Rotation angle in degrees, +ve is counter-clockwise
    
    bb = np.array(((bb[0],bb[1]),(bb[2],bb[1]),(bb[2],bb[3]),(bb[0],bb[3]))) #Get all 4 coordinates of the box
    
    img = cv2.imread(img_path) #Read the img
    
    center = (img.shape[0]//2,img.shape[1]//2) #Get the center of the image
    
    rotMat = cv2.getRotationMatrix2D(center,angle,1.0) #Get the rotation matrix, its of shape 2x3
    
    img_rotated = cv2.warpAffine(img,rotMat,img.shape[1::-1]) #Rotate the image
    
    bb_rotated = np.vstack((bb.T,np.array((1,1,1,1)))) #Convert the array to [x,y,1] format to dot it with the rotMat
    bb_rotated = np.dot(rotMat,bb_rotated).T #Perform Dot product and get back the points in shape of (4,2)
    
    #Plot the original image and bb
    plt.imshow(img)
    plt.plot(
        np.append(bb[:,0],bb[0,0]),
        np.append(bb[:,1],bb[0,1])
    )
    plt.show()
    
    #Plot the rotated image and bb
    plt.imshow(img_rotated)
    plt.plot(
        np.append(bb_rotated[:,0],bb_rotated[0,0]),
        np.append(bb_rotated[:,1],bb_rotated[0,1])
    )
    plt.show()

Answer 2

Here is one way in Python/OpenCV.

• Read the input
• Convert to HSV
• Do color thresholding on the green box
• Get the outer contour
• Print the bounding box
• Rotate the image by 10 deg clocwise
• Convert that image to HSV
• Do color thresholding on the rotated green box
• Get the outer contour
• Create a black image with the white filled contour
• Get the white pixel coordinates
• Get the minAreaRect from the coordinates
• Get the vertices of the rotated rectangle
• Draw the rotated rectangle outline on the rotated image

Input:

import cv2
import numpy as np
from scipy import ndimage

# load image
img = cv2.imread("berry.png")

# convert to hsv
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# threshold using inRange or green
range1 = (20,200,170)
range2 = (80,255,255)
thresh = cv2.inRange(hsv,range1,range2)

# get bounding box coordinates from the one outer contour
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
x,y,w,h = cv2.boundingRect(contours[0])
print("bounding_box(x,y,w,h):",x,y,w,h)

# rotate image by 10 degree clockwise
rotated = img.copy()
rotated = ndimage.rotate(img, -10, cval=255)


# convert rotated to hsv
hsv_rotated = cv2.cvtColor(rotated, cv2.COLOR_BGR2HSV)

# threshold using inRange or green
range1 = (20,200,170)
range2 = (80,255,255)
thresh_rotated = cv2.inRange(hsv_rotated,range1,range2)

# get bounding box coordinates from the one outer contour
contours = cv2.findContours(thresh_rotated, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]

# draw white filled contour on black background
mask = np.zeros_like(thresh_rotated)
cv2.drawContours(mask, [contours[0]], -1, (255), -1)

# get coordinates of white pixels in mask
coords = np.column_stack(np.where(mask.transpose() > 0))

# get rotated rectangle
rotrect = cv2.minAreaRect(coords)

# rotated rectangle box points
box = np.int0(cv2.boxPoints(rotrect))
print("rotate_box_corners:\n",box)

# draw rotated rectangle on rotated image
result = rotated.copy()
cv2.polylines(result, [box], True, (0,0,255), 1)


# write result to disk
cv2.imwrite("berry_thresh.png", thresh)
cv2.imwrite("berry_rotated.png", rotated)
cv2.imwrite("berry_thresh_rotated.png", thresh_rotated)
cv2.imwrite("berry_mask.png", mask)
cv2.imwrite("berry_rotated_box.png", result)

# display results
cv2.imshow("THRESH", thresh)
cv2.imshow("ROTATED", rotated)
cv2.imshow("THRESH_ROT", thresh_rotated)
cv2.imshow("MASK", mask)
cv2.imshow("RESULT", result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Threshold of green lines in input:

Rotated input:

Threshold of green lines in rotated image:

Filled threshold:

Result showing rotated rectangle on rotated image:

Input Bounding Box:

bounding_box(x,y,w,h): 12 13 212 124

Output Vertices:

rotate_box_corners:
 [[222 172]
 [ 14 136]
 [ 35  14]
 [243  51]]