Remove the black line surrounding text in opencv

Question

I am trying to remove the black lines surrounding the text if present any. My purpose is to just have enough portion of the image to extract each character in it. The additional black lines are noise when i am trying to extract characters.

I have tried using floodfill in opencv but the image contains some white pixels before the black line starts in the upper left corner. So it hasn't been fruitful. I tried cropping by means of finding contours but even that does not work. The image is as follows:

import cv2
import numpy as np

img = cv2.imread('./Cropped/22.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
_,thresh = cv2.threshold(gray,1,255,cv2.THRESH_BINARY)
contours,hierarchy = cv2.findContours(thresh,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
cnt = contours[0]
x,y,w,h = cv2.boundingRect(cnt)
crop = img[y:y+h,x:x+w]

cv2.imshow('Image',img)
cv2.imshow('Cropped Image',crop)
cv2.waitKey(0)

and using floodfill

img = cv2.imread('./Cropped/22.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# threshold the gray image to binarize, and negate it
gray = cv2.bitwise_not(gray)
w = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, \
                          cv2.THRESH_BINARY, 15, -2)

# find external contours of all shapes
contours,h = cv2.findContours(bw, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

# create a mask for floodfill function, see documentation
h,w,_ = img.shape
mask = np.zeros((h+2,w+2), np.uint8)

# determine which contour belongs to a square or rectangle
for cnt in contours:
    poly = cv2.approxPolyDP(cnt, 0.02*cv2.arcLength(cnt,True),True)
    if len(poly) == 4:
        # if the contour has 4 vertices then floodfill that contour with black color
        cnt = np.vstack(cnt).squeeze()
        _,binary,_,_ = cv2.floodFill(bw, mask, tuple(cnt[0]), 0)
# convert image back to original color
binary = cv2.bitwise_not(binary)        

cv2.imshow('Image', binary)
cv2.waitKey(0)
cv2.destroyAllWindows()

The results in the two cases are as follows

But there appears to be no change and

which does not remove any borders. The ideas of both the codes was obtained from stack overflow answers to similar questions.

EDIT

I approached the solution as mentioned in the comment by @rayryeng. However when i enter the cropped up image for number extraction i get these images and a wrong result. I guess some noisy pixels aren't getting removed. This is the original image Original Image. The thresholded image is Thresholding Image . The contours extracted are as follows First contour, Second contour, Third contour, Fourth contour. If there could be a generalised solution to this, it would be great.

Answer 1

Notice that the black lines occupy a significantly smaller area than the text itself. In addition, we can leverage the fact that the text is very tightly close together. Therefore, one thing I can suggest is to merge the text blobs together so they're one large blob. Using the fact that there is one blob at the top and one blob at the bottom, once we detect the contours we should hopefully we will have three blobs, then choose the blob that has the largest area and form a bounding rectangle around that.

You can join binary blobs together with morphological closing, then find the contours and extract their areas. As additional processing, let's also dilate the blobs slightly so that we can see more of the background of the text before we crop. After, choose the blob with the largest area and crop.

Take note that I had to not only threshold your image, but perform an inverse threshold where black areas become white and vice-versa. In addition, I had to change your threshold from 1 to 128. Setting a threshold of 1 for an unsigned 8-bit image means that you're creating a binary image where almost everything would be white. With images like these, you have to increase the tolerance. Finally, cv2.findContours has slightly different ways of being called between OpenCV 2.4.x and OpenCV 3.x. In summary, there's an additional output from the method which is the source image that you provided to the method, so you can safely ignore this.

import cv2
import numpy as np

img = cv2.imread('MgPg8.jpg') # Image saved offline on my computer
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
_,thresh = cv2.threshold(gray,128,255,cv2.THRESH_BINARY_INV) # Change

# Perform morphological closing
out = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, 255*np.ones((11, 11), dtype=np.uint8))
# Perform dilation to expand the borders of the text to be sure
out = cv2.dilate(thresh, 255*np.ones((11, 11), dtype=np.uint8))

# For OpenCV 3.0
_,contours,hierarchy = cv2.findContours(out,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE) # Change
# For OpenCV 2.4.x
# contours,hierarchy = cv2.findContours(out,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

# Find the area made by each contour
areas = [cv2.contourArea(c) for c in contours]

# Figure out which contour has the largest area
idx = np.argmax(areas)

# Choose that contour, then get the bounding rectangle for this contour
cnt = contours[idx]
x,y,w,h = cv2.boundingRect(cnt)

# Crop
crop = img[y:y+h,x:x+w]

cv2.imshow('Image',img)
cv2.imshow('Thresholded Image',thresh)
cv2.imshow('Closed Image',out)
cv2.imshow('Cropped', crop)
cv2.imwrite('thresh.png', thresh)
cv2.imwrite('binary.png', out)
cv2.imwrite('crop.png', crop)
cv2.waitKey(0)
cv2.destroyAllWindows()

I get the following for the thresholded image, the morphologically processed image, and finally the cropped image: