how to detect rectangle shape object using python

Question

I am doing a project in python.I want to detect rectangle shape object by opening the webcam using python.I had tried it but I didn't get accurate Answer.I show the object in front of webcam If any finger touched our object it doesn't recognize our object.please anyone can help me.Thanks in Advance:) Here is my code: py:

import math
import numpy as np
import cv2

#dictionary of all contours
contours = {}
#array of edges of polygon
approx = []
#scale of the text
scale = 2
#camera
cap = cv2.VideoCapture(0)
print("press q to exit")

# Define the codec and create VideoWriter object
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('output.avi',fourcc, 20.0, (640,480))

#calculate angle
def angle(pt1,pt2,pt0):
    dx1 = pt1[0][0] - pt0[0][0]
    dy1 = pt1[0][1] - pt0[0][1]
    dx2 = pt2[0][0] - pt0[0][0]
    dy2 = pt2[0][1] - pt0[0][1]
    return float((dx1*dx2 + dy1*dy2))/math.sqrt(float((dx1*dx1 + dy1*dy1))*(dx2*dx2 + dy2*dy2) + 1e-10)

while(cap.isOpened()):
    #Capture frame-by-frame
    ret, frame = cap.read()
    if ret==True:
        #grayscale
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        #Canny
        canny = cv2.Canny(frame,80,240,3)

        #contours
        canny2, contours, hierarchy = cv2.findContours(canny,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
        for i in range(0,len(contours)):
            #approximate the contour with accuracy proportional to
            #the contour perimeter
            approx = cv2.approxPolyDP(contours[i],cv2.arcLength(contours[i],True)*0.02,True)

            #Skip small or non-convex objects
            if(abs(cv2.contourArea(contours[i]))<100 or not(cv2.isContourConvex(approx))):
                continue

            x,y,w,h = cv2.boundingRect(contours[i])
            vtc = len(approx)
            if(vtc==4):
                cv2.putText(frame,'RECTANGLE',(x,y),cv2.FONT_HERSHEY_SIMPLEX,scale,(255,255,255),2,cv2.LINE_AA)

        #Display the resulting frame
        out.write(frame)
        cv2.imshow('frame',frame)
        cv2.imshow('canny',canny)
        if cv2.waitKey(1) == 1048689: #if q is pressed
            break

#When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

Here is my output:

Non Working output

Working output

Answer 1

You currently have this part in your code:

        #Skip small or non-convex objects
        if(abs(cv2.contourArea(contours[i]))<100 or not(cv2.isContourConvex(approx))):
            continue

        x,y,w,h = cv2.boundingRect(contours[i])
        vtc = len(approx)
        if(vtc==4):
            cv2.putText(frame,'RECTANGLE',(x,y),cv2.FONT_HERSHEY_SIMPLEX,scale,(255,255,255),2,cv2.LINE_AA)

here it is possible to create a rectangle around the contour and compare the areas. For that it is possible to use boundingRect, however, your phone can be slightly angled, so minAreaRect is better for this. It will return ((x,y), (w,h), angle) you care is the (w,h) part since the area is w*h. You already know hot to get the actual area of the contour, since it is in your code.

At the end the code should look like this:

        #Skip small or non-convex objects
        if(abs(cv2.contourArea(contours[i]))<100 or not(cv2.isContourConvex(approx))):
            continue

        x,y,w,h = cv2.boundingRect(contours[i])
        vtc = len(approx)
        rect = cv2.minAreaRect(contours[i])
        rectArea = rect[1][0] * rect[1][1]
        contourArea = cv2.contourArea(contours[i])
        # now it will check if the difference is less than 10% of the rect area
        if vtc==4 or abs(rectArea - contourArea) < rectArea * .10:
            cv2.putText(frame,'RECTANGLE',(x,y),cv2.FONT_HERSHEY_SIMPLEX,scale,(255,255,255),2,cv2.LINE_AA)

Probably this will work, but you may need to adjust the threshold (I used 10% of rectArea). Even the 4 point check can be omitted, If it is a rectangle it will have a perfect fit (rectarea-contourarea) = 0.

I hope this helps, however this is a simple way. More possible answers are also valid for this. You can even think it with machine learning algorithms, or rectangle fitting algorithms.