CODEWITHSUNDEEP
pythonpython-3.xopencvimage-processingopencv3.0
Alan Jones
Alan Jones

Reputation: 462

How to detect the text above lines using OpenCV in Python

I am interested in detecting lines (which I managed to figure out using hough transform) and the text above it.

My test image is below: Test Image

The code I have written is below. ( I have edited so that I can loop through the coordinates of each line)

import cv2
import numpy as np

img=cv2.imread('test3.jpg')
#img=cv2.resize(img,(500,500))
imgGray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
imgEdges=cv2.Canny(imgGray,100,250)
imgLines= cv2.HoughLinesP(imgEdges,1,np.pi/180,230, minLineLength = 700, maxLineGap = 100)
imgLinesList= list(imgLines)

a,b,c=imgLines.shape
line_coords_list = []
for i in range(a):
    line_coords_list.append([(int(imgLines[i][0][0]), int(imgLines[i][0][1])), (int(imgLines[i][0][2]), int(imgLines[i][0][3]))])

print(line_coords_list)#[[(85, 523), (964, 523)], [(85, 115), (964, 115)], [(85, 360), (964, 360)], [(85, 441), (964, 441)], [(85, 278), (964, 278)], [(85, 197), (964, 197)]]

roi= img[int(line_coords_list[0][0][1]): int(line_coords_list[0][1][1]), int(line_coords_list[0][0][0]) : int(line_coords_list[0][1][0])]
print(roi) # why does this print an empty list?
cv2.imshow('Roi NEW',roi)

Now I just don't know how to detect the region of interest above those lines. Is it possible to say crop out each line and have images say roi_1 , roi_2 , roi_n where each roi is the text above the first line, the text above the second line etc?

I would like the output to be something like this.

Upvotes: 2

Views: 5233

Answers (2)

fmw42
fmw42

Reputation: 53089

Here is one way to do that in Python/OpenCV.

  • Read the input
  • Convert to gray
  • Threshold (OTSU) so that text is white on black background
  • Apply morphology dilate with horizontal kernel to blur text in a line together
  • Apply morphology open with a vertical kernel to remove the thin lines from the dotted lines
  • Get the contours
  • Find the contour that has the lowest Y bounding box value (top-most box)
  • Draw all the bounding boxes on the input except for the topmost one
  • Save results

Input:

enter image description here

import cv2
import numpy as np

# load image
img = cv2.imread("text_above_lines.jpg")

# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# threshold the grayscale image
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]

# use morphology erode to blur horizontally
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (151, 3))
morph = cv2.morphologyEx(thresh, cv2.MORPH_DILATE, kernel)

# use morphology open to remove thin lines from dotted lines
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 17))
morph = cv2.morphologyEx(morph, cv2.MORPH_OPEN, kernel)

# find contours
cntrs = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cntrs = cntrs[0] if len(cntrs) == 2 else cntrs[1]

# find the topmost box
ythresh = 1000000
for c in cntrs:
    box = cv2.boundingRect(c)
    x,y,w,h = box
    if y < ythresh:
        topbox = box
        ythresh = y

# Draw contours excluding the topmost box
result = img.copy()
for c in cntrs:
    box = cv2.boundingRect(c)
    if box != topbox:
        x,y,w,h = box
        cv2.rectangle(result, (x, y), (x+w, y+h), (0, 0, 255), 2)

# write result to disk
cv2.imwrite("text_above_lines_threshold.png", thresh)
cv2.imwrite("text_above_lines_morph.png", morph)
cv2.imwrite("text_above_lines_lines.jpg", result)

#cv2.imshow("GRAY", gray)
cv2.imshow("THRESH", thresh)
cv2.imshow("MORPH", morph)
cv2.imshow("RESULT", result)
cv2.waitKey(0)
cv2.destroyAllWindows()


Thresholded image:

enter image description here

Morphology image:

enter image description here

Result:

enter image description here

Upvotes: 5

lenik
lenik

Reputation: 23508

You have detected the lines. Now you have to split your image into regions between the lines using y coordinate and then search for the black pixels (words) on the white background (paper).

Building a histogram along the x and y axes will likely give you the area of interest you're looking for.

Just to answer your questions in the comments, for example, if you have an image img and area of the interest with y coordinates (100,200) spanning the whole width of the image, you may crop that area down and search for anything there like this:

cropped = img[100:200,5:-5]  # crop a few pixels off in x-direction just in case

Now the search:

top, left = 10000, 10000
bottom, right = 0, 0
for i in range(cropped.shape[0]) :
    for j in range(cropped.shape[1]) :
        if cropped[i][j] < 200 :    # black?
            top = min( i, top)
            bottom = max( i, bottom)
            left = min( j, left)
            right = max( j, right)

Or something along the lines...

Upvotes: 1

Related Questions