pupil detection in OpenCV & Python

Question

I am doing pupil detection for my school project. It's my first time working with OpenCV and Python, using Python version 3.4.2 and OpenCV 3.1.0.

I am using the Raspberry Pi NoIR camera, and I am getting good images.

But i can't detect a pupil nicely (because of glint, lashes and shadows. I refer to some code on the web and the following is part of that code.

...

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))

# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):

    image = frame.array
    cv2.imshow("image", image)


    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    retval, thresholded = cv2.threshold(gray, 80, 255, 0)
    cv2.imshow("threshold", thresholded)

    closed = cv2.erode(cv2.dilate(thresholded, kernel, iterations=1), kernel, iterations=1)
    #closed = cv2.morphologyEx(close, cv2.MORPH_CLOSE, kernel)

    cv2.imshow("closed", closed)

    thresholded, contours, hierarchy = cv2.findContours(closed, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)

    drawing = np.copy(image)
    cv2.drawContours(drawing, contours, -1, (255, 0, 0), 2)

    for contour in contours:

        area = cv2.contourArea(contour)
        bounding_box = cv2.boundingRect(contour)

        extend = area / (bounding_box[2] * bounding_box[3])

        # reject the contours with big extend
        if extend > 0.8:
            continue

        # calculate countour center and draw a dot there
        m = cv2.moments(contour)
        if m['m00'] != 0:
            center = (int(m['m10'] / m['m00']), int(m['m01'] / m['m00']))
            cv2.circle(drawing, center, 3, (0, 255, 0), -1)

        # fit an ellipse around the contour and draw it into the image
        try:
            ellipse = cv2.fitEllipse(contour)
            cv2.ellipse(drawing, box=ellipse, color=(0, 255, 0))
        except:
            pass

    # show the frame
    cv2.imshow("Drawing", drawing)

    ...

Input image:

Output image:

How can I remove the parts of the image that are not related to the pupil, as shown above?

In addition to answers, any hints are also welcome.

Answer 1

Some years ago I realized a very similar project. To everything added above I can suggest one little trick.

As you can see you have two reflections from any light source. One on the surface of the eye and second reflection on the surface of the glass.

If you will remove the glass (if it's possible) you will have a very bright reflection almost in the center of pupil. In that case you can ignore all objects without this bright reflection. This also will help you to find a position of eye in the space near camera

Answer 2

There are several things you can do. How well they work depends on how much variation there is in the images you want to apply the algorithm on. You could make several assumptions and then discard all the candidates that do not meet them.

remove small detections

At first I would consider removing candidates that are too small by adding this line at the beginning of your loop:

if area < 100:
    continue

The threshold was chosen randomly and worked well for this particular image. It removed almost all the false detection. Only the biggest one remains. But you have to check it against your other images and adapt it to your needs.

remove detections that are not round

Another assumption you can make is that pupils are usually round and you can remove every detection that is not 'round' enough. One simple measure of roundness is to look at the ratio of circumference to area.

circumference = cv2.arcLength(contour,True)
circularity = circumference ** 2 / (4*math.pi*area)

The circularity is about 2.72 for the shadow in the right side and 1.31 for the pupil.

improving roundness

You notice, that the contour of your pupil is not perfectly round because of reflections. You can improve this by computing the convex hull of the contours.

contour = cv2.convexHull(contour)

If you do that before computing the area and circumference you get circularity values of 1.01 and 1.37. (A perfect circle has a circularity of 1) This means the defect from the reflection was almost perfectly repaired. This might not be necessary in this case but could be useful in cases with more reflections.