CODEWITHSUNDEEP
pythonopencvimage-processingextractimage-recognition
Mithril
Mithril

Reputation: 13728

opencv findContours miss some area.[ not get all correct bounding boxes ]

I am new to opencv, start to learn it by extract char from simple captcha. After some effort, I got findContours and some method to clean the image, sometimes worked, but not more often.

For example:

  1. I have a original image(already scale to a large size): enter image description here

  2. convert to grayscale and use cv2.threshold clean: enter image description here

  3. use cv2.findContours to get bounding boxes:

enter image description here W only cover a half, and not get b.

My code:

from StringIO import StringIO
import string

from PIL import Image
import requests
import cv2
import numpy as np
import matplotlib.pyplot as plt

def get_ysdm_captcha():
    url = 'http://www.ysdm.net/common/CleintCaptcha'
    r = requests.get(url)
    img = Image.open(StringIO(r.content))
    return img

def scale_image(img, ratio):
    return img.resize((int(img.width*ratio), int(img.height*ratio)))

def draw_rect(im):
    im = np.array(im)

    if len(im.shape) == 3 and im.shape[2] == 3:
        imgray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
    else:
        imgray = im

    #plt.imshow(Image.fromarray(imgray), 'gray')
    pilimg = Image.fromarray(imgray)
    ret,thresh = cv2.threshold(imgray,127,255,0)

    threimg = Image.fromarray(thresh)

    plt.figure(figsize=(4,3))
    plt.imshow(threimg, 'gray')
    plt.xticks([]), plt.yticks([])

    contours, hierarchy = cv2.findContours(np.array(thresh),cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
    areas = []

    for c in contours:
        rect = cv2.boundingRect(c)
        area = cv2.contourArea(c)
        areas.append(area)
        x,y,w,h = rect

        if area > 2000 or area < 200 : continue

        cv2.rectangle(thresh,(x,y),(x+w,y+h),(0,255,0),1)
        plt.figure(figsize=(1,1))
        plt.imshow(threimg.crop((x,y,x+w,y+h)), 'gray')
        plt.xticks([]), plt.yticks([])

    plt.figure(figsize=(10,10))

    plt.figure()
    plt.imshow(Image.fromarray(thresh), 'gray')
    plt.xticks([]), plt.yticks([])


image = get_ysdm_captcha()
im = scale_image(image, 3)
im = np.array(im)

imgray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
imgray = cv2.GaussianBlur(imgray,(5,5),0)
# im = cv2.medianBlur(imgray,9)
# im = cv2.bilateralFilter(imgray,9,75,75)

draw_rect(imgray)

I tried my best to write above code. The solutions I imagine is:

  1. find was there any way to tell cv2.findContours I need 4 bounding boxes in some size
  2. tried some different parameter (I tried all from http://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=findcontours#findcontours , but still not work)

Now I'm stuck , have no idea how to improve cv2.findContours...

Upvotes: 0

Views: 1477

Answers (2)

Rob Audenaerde
Rob Audenaerde

Reputation: 20019

You can use morphological operations to modify your image and fill the gaps, for example erode and dilate

See here: http://docs.opencv.org/2.4/doc/tutorials/imgproc/erosion_dilatation/erosion_dilatation.html

Original:

enter image description here

Dilated:

enter image description here

By the way: I would implement a HSV separation step in the original image, removing all the 'white/grey/black' content (low saturation). This will reduce the number of specks. Do this before converting to grayscale.

Here is the result filtering on: saturation > 90 enter image description here

Final result: (Added a blur step before)

enter image description here

Also, if there always a gradient, you could detect this and filter out even more colors. But that is a bit much if you just started image processing ;)

Upvotes: 2

Tides
Tides

Reputation: 121

findCountours works properly as it finds all connected component of your image. Your area condition is what is probably avoiding you to get a bounding box around letter b, for instance. Of course, if you put a bounding box around each connected component you will not end up with a bounding box around each character because you have many holes in your letters.

If you want to segment the letters, I would first try to play with opening operations (because your letters are black on a white background, it would be closing if it was the opposite) in order to fill the holes that you have in your letters. Then I would project vertically the pixels and analyze the shape that you get. If you find the valley points in this projected shape you will get the vertical limits between characters. You can do the same horizontally to get the upper and bottom limits of your chars. This approach will only work if the text is horizontal. If it is not, you should find the main axis angle of your string and you could rotate the image accordingly. To find the main axis angle you could fit an ellipse to your text and find its main axis angle or you could keep rotating your image by a certain angle until your horixontal projection is maximum.

Upvotes: 0

Related Questions