CODEWITHSUNDEEP
pythonpython-3.xopencvpython-imaging-library
Spencer McMurray
Spencer McMurray

Reputation: 33

Recognizing handwritten digits off a scanned image

I'm trying to read all of the handwritten digits in the scanned image here

I tried looking through pixel-by-pixel using PIL, cropping the sub-images, then feeding them through a neural network, but the regions that were being cropped never quite lined up and led to a lot of inaccuracy.

I've also tried using OpenCV to find all the grey squares, then crop the images and feed them through a neural network, but I couldn't seem to get it to find all or even only miss a few; it would miss ~30% of squares. (I'm not very experienced with OpenCV, so I could be messing something up)

So I'm just looking for a potential idea/solution for this problem, so any suggestions would be appreciated, thanks in advance!

Upvotes: 3

Views: 1149

Answers (1)

Howard GENG
Howard GENG

Reputation: 1105

I assume the input image name is "sqaures.jpg"

First of all, import required libraries and load image in both RGB and Gray format:

import cv2
import numpy as np

image = cv2.imread("squares.jpg", 1)
image_gray = cv2.imread("squares.jpg", 0)

Then, we perform a simple operation to clean some noise from the input image using np.where() function:

image_gray = np.where(image_gray > 240, 255, image_gray)
image_gray = np.where(image_gray <= 240, 0, image_gray)

Because we want to grab the whole square regions from the image. We need to blur the image a little bit before performing the adaptive thresholding method:

image_gray = cv2.blur(image_gray, (5, 5))
im_th = cv2.adaptiveThreshold(image_gray, 255, 
                              cv2.ADAPTIVE_THRESH_GAUSSIAN_C, 
                              cv2.THRESH_BINARY, 115, 1)

kernal = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))
im_th = cv2.morphologyEx(im_th, cv2.MORPH_OPEN, kernal, iterations=3)

Use contour detection in OpenCV to find all possible regions:

_, contours, _ = cv2.findContours(im_th.copy(), cv2.RETR_LIST, 
                                  cv2.CHAIN_APPROX_SIMPLE)

contours = sorted(contours, key=cv2.contourArea, reverse=True)
contours.remove(contours[0])  #remove the biggest contour

Finally, try to find the potential square regions based on the ratio of height and width:

square_rects = []
square_areas = []
for i, cnt in enumerate(contours):
    (x, y, w, h) = cv2.boundingRect(cnt)
    ar = w / float(h)
    if 0.9 < ar < 1.1:
        square_rects.append(((x,y), (x+w, y+h)))
        square_areas.append(w*h)  #store area information

We need to remove anything that is too small from the list by doing the follows:

import statistics
median_size_limit= statistics.median(square_areas) * 0.8
square_rects = [rect for i, rect in enumerate(square_rects)
                    if square_areas[i] > median_size_limit]

You can visually check the output by drawing all the rectangles on the original image:

for rect in square_rects:
    cv2.rectangle(image, rect[0], rect[1], (0,255,0), 2)

cv2.imwrite("_output_image.png", image)

cv2.imshow("image", image)
cv2.waitKey()

You can use "square_rects" to locate all the squares and crop them from the original image.

The following is the preview of the final result. Preview of Final Result

Cheers.

Upvotes: 6

Related Questions