How to detect clock hands with hough lines detection

Question

I want to get the time from an analog clock. Right now I'm stuck a bit, I managed to get the segmented image (altough I couldn't remove the bottom part of it...), and did a Canny detection. The problem I have is, well the bottom part I couldn't remove, and the detection of the clock hands. My goal is to detect the hands in a way I can calculate the angles and then the time from those angles. I know that I need Hough Line Transform, but I don't really understand how it works, how to set the parameters.

The original, segmented and the Canny detected pictures:

This is the code I'm using to get there:

img = cv2.imread('clock.jpg')
cv2.imshow('img', img)
cv2.waitKey(0)

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blur = cv2.medianBlur(gray, 5)
cv2.imshow('blur', blur)
cv2.waitKey(0)

circles = cv2.HoughCircles(blur, cv2.HOUGH_GRADIENT, 1, 20, param1=20, param2=100, minRadius=0, maxRadius=0)
detected_cricles = np.uint16(np.around(circles))
circle = detected_cricles[0][0]

x = circle[0]
y = circle[1]
r = circle[2]

rect = (x - r, y - r, x+r, y+(r-10))
mask = np.zeros(img.shape[:2], dtype = np.uint8)
bgdModel = np.zeros((1,65), np.float64)
fgdModel = np.zeros((1,65), np.float64)

cv2.grabCut(img, mask, rect, bgdModel, fgdModel, 1, cv2.GC_INIT_WITH_RECT)
mask2 = np.where((mask == 1) + (mask == 3), 255, 0).astype('uint8')
segmented = cv2.bitwise_and(img, img, mask=mask2)
cv2.imshow('segmented', segmented)
cv2.waitKey(0)

blur = cv2.GaussianBlur(segmented, (11,11), 0)
cv2.imshow('blur2', blur)
cv2.waitKey(0)

canny = cv2.Canny(blur, 30, 150, None, 3)
cv2.imshow('canny', canny)
cv2.waitKey(0)

Answer 1

Here is one way using HoughLinesP in Python/OpenCV. The approach uses thresholding, contours and thinning before getting the Hough Lines. I will leave it to you to compute the angles from the line end points.

Input:

import cv2
import numpy as np
from skimage.morphology import skeletonize

# Read image
img = cv2.imread('clock.jpg')
hh, ww = img.shape[:2]

# convert to gray
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

# threshold
thresh = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY)[1]

# invert so shapes are white on black background
thresh = 255 - thresh

# get contours and save area
cntrs_info = []
contours = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
index=0
for cntr in contours:
    area = cv2.contourArea(cntr)
    cntrs_info.append((index,area))
    index = index + 1

# sort contours by area
def takeSecond(elem):
    return elem[1]
cntrs_info.sort(key=takeSecond, reverse=True)

# get third largest contour
arms = np.zeros_like(thresh)
index_third = cntrs_info[2][0]
cv2.drawContours(arms,[contours[index_third]],0,(1),-1)

#arms=cv2.ximgproc.thinning(arms)
arms_thin = skeletonize(arms)
arms_thin = (255*arms_thin).clip(0,255).astype(np.uint8)

# get hough lines and draw on copy of input
result = img.copy()
lineThresh = 15
minLineLength = 20
maxLineGap = 100
max
lines = cv2.HoughLinesP(arms_thin, 1, np.pi/180, lineThresh, None, minLineLength, maxLineGap)

for [line] in lines:
    x1 = line[0]
    y1 = line[1]
    x2 = line[2]
    y2 = line[3]
    cv2.line(result, (x1,y1), (x2,y2), (0,0,255), 2)   

# save results
cv2.imwrite('clock_thresh.jpg', thresh)
cv2.imwrite('clock_arms.jpg', (255*arms).clip(0,255).astype(np.uint8))
cv2.imwrite('clock_arms_thin.jpg', arms_thin)
cv2.imwrite('clock_lines.jpg', result)

cv2.imshow('thresh', thresh)
cv2.imshow('arms', (255*arms).clip(0,255).astype(np.uint8))
cv2.imshow('arms_thin', arms_thin)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Thresholded image:

Contour of arms:

Thinned (skeleton):

Hough Line Segments on input:

Answer 2

Here's another possible solution. We will try to segment the clocks hands and run them through Hough's line transform to detect the lines. Now, this detection will yield all the possible straight lines that pass through the clock hands' pixels - producing multiple lines. You can try to play with the line transform parameters to narrow the result to the target lines, but you will probably end up with a cluster of lines. I will try to cluster these lines using K-Means to get only two lines regardless of the output of Hough's line transform. These are the steps:

1. Get a binary mask of the image to isolate the clock hands
2. Apply some morphology to get rid of the noise
3. Run the binary mask through Hough's line detection
4. Use K-means on the multiple lines to get only 2 (average) lines (one per clock hand)

Let's see the code:

# Imports
import cv2
import numpy as np

# Read image
imagePath = "D://opencvImages//"
inputImage = cv2.imread(imagePath+"orFGl.jpg")

# Store deep copy for results:
originalImg = inputImage.copy()

# Convert BGR back to grayscale:
grayInput = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)

# Threshold via Otsu + bias adjustment:
threshValue, binaryImage = cv2.threshold(grayInput, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)

The first bit is trivial and produces this binary mask:

We can get rid of the small elements via some morphology. Let's apply an erosion followed by a dilation to filter everything but the larger components - the clock hands:

# Set morph operation iterations:
opIterations = 1

# Get the structuring element:
structuringElement = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))

# Perform Erode:
erodeImg = cv2.morphologyEx(binaryImage, cv2.MORPH_ERODE, structuringElement, None, None, opIterations, cv2.BORDER_REFLECT101)
# Perform Dilate:
dilateImg = cv2.morphologyEx(erodeImg, cv2.MORPH_DILATE, structuringElement, None, None, opIterations, cv2.BORDER_REFLECT101)

This produces this image:

Very nice, almost all the noise is gone. Let's run this directly through the line detection and check out what kind of results we get. Additionally, I've prepared some lists to store every starting (x1, y1) and ending (x2, y2) point of the lines:

# Set HoughLinesP parameters:
lineThresh = 50
minLineLength = 20
maxLineGap = 100
# Run the line detection:
lines = cv2.HoughLinesP(dilateImg, 1, np.pi/180, lineThresh, None, minLineLength, maxLineGap)

# Prepare some lists to store every coordinate of the detected lines:
X1 = []
X2 = []
Y1 = []
Y2 = []

# Store and draw the lines:
for [currentLine] in lines:

    # First point:
    x1 = currentLine[0]
    y1 = currentLine[1]
    X1.append(x1)
    Y1.append(y1)

    # Second point:
    x2 = currentLine[2]
    y2 = currentLine[3]
    X2.append(x2)
    Y2.append(y2)

    # Draw the lines:
    cv2.line(originalImg, (x1,y1), (x2,y2), (0,0,255), 2)
    cv2.imshow("Lines", originalImg)
    cv2.waitKey(0)

This is the result:

As you can see, there are multiple lines. Luckily, these lines are clustered in two very discernible groups: the left hand and the right hand. If we cluster the four coordinates into two groups, we can get the average starting and ending points of each hand. This can be done by applying a clustering algorithm, in this case K-Means. K-means will need four arrays holding the data to produce two cluster centers. Before giving it our data we need to reshape it the way K-means expects it:

# Reshape the arrays for K-means
X1 = np.array(X1)
Y1 = np.array(Y1)
X2 = np.array(X2)
Y2 = np.array(Y2)

X1dash = X1.reshape(-1,1)
Y1dash = Y1.reshape(-1,1)
X2dash = X2.reshape(-1,1)
Y2dash = Y2.reshape(-1,1)

# Stack the data
Z = np.hstack((X1dash, Y1dash, X2dash, Y2dash))

# K-means operates on 32-bit float data:
floatPoints = np.float32(Z)

# Set the convergence criteria and call K-means:
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
# Set the desired number of clusters
K = 2
ret, label, center = cv2.kmeans(floatPoints, K, None, criteria, 10, cv2.KMEANS_RANDOM_CENTERS)

The results are in the center array. Here we gave out final pair of lines. Let's loop through it and draw them on the original image:

# Loop through the center points
# and draw the lines:
for p in range(len(center)):

    # Get line points:
    print(center[p])
    x1 = int(center[p][0])
    y1 = int(center[p][1])
    x2 = int(center[p][2])
    y2 = int(center[p][3])

    cv2.line(originalImg, (x1, y1), (x2, y2), (0, 255, 0), 1)
    cv2.imshow("Lines", originalImg)
    cv2.waitKey(0)

This is the final pair of lines (in green):