Python + OpenCV color segmentation using Kmeans

Question

I am trying to apply the kmeans from opencv in order to segment the image in HSV color space.

def leftOffset(src, p_countours):
    height, width, size = src.shape

    p_width = width/p_countours
    o_left = src[0:height, 0:p_width]

    HSV_img = cv2.cvtColor(o_left, cv2.COLOR_BGR2HSV)
    hue = HSV_img[0]
    hue = np.float32(HSV_img)

    # Define criteria = ( type, max_iter = 10 , epsilon = 1.0 )
    criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)

    # Set flags (Just to avoid line break in the code)
    flags = cv2.KMEANS_RANDOM_CENTERS

    # Apply KMeans
    compactness,labels,centers = cv2.kmeans(hue,2,criteria,10,flags)

    centers = np.uint8(centers)
    res = centers[labels.flatten()]
    res2 = res.reshape((hue.shape))
    cv2.imshow("o_left", hue)
    cv2.waitKey(0)

I am now able to apply the kmeans algorithm to the HSVImage[0] with K=2, and how can I get a image like threshold according to the result?

Thanks

To clarify the question: I have color-based captchas, and I want to segment each digits.

The image is like

I am going to use k-means method to find out the dominant color and segment the digits inside.

Answer 1

May I suggest a conventional alternative? I you get rid of the very dark and bright regions first, you may be able to simply rely on the most frequent value of the hue component calculated from the histogram.

Mind that the borders of the numbers will never be absolutely exact, since colours are similar in the surrounding.

Furthrmore, you could select the maximum blob only (according to size) to suppress remaining small blobs outside.

Results:

Code:

import cv2
import numpy as np
from matplotlib import pyplot as plt


img = cv2.imread('image1.jpg')

#get rid of very bright and very dark regions
delta=30
lower_gray = np.array([delta, delta,delta])
upper_gray = np.array([255-delta,255-delta,255-delta])
# Threshold the image to get only selected
mask = cv2.inRange(img, lower_gray, upper_gray)
# Bitwise-AND mask and original image
res = cv2.bitwise_and(img,img, mask= mask)

#Convert to HSV space
HSV_img = cv2.cvtColor(res, cv2.COLOR_BGR2HSV)
hue = HSV_img[:, :, 0]

#select maximum value of H component from histogram
hist = cv2.calcHist([hue],[0],None,[256],[0,256])
hist= hist[1:, :] #suppress black value
elem = np.argmax(hist)
print np.max(hist), np.argmax(hist)

tolerance=10
lower_gray = np.array([elem-tolerance, 0,0])
upper_gray = np.array([elem+tolerance,255,255])
# Threshold the image to get only selected
mask = cv2.inRange(HSV_img, lower_gray, upper_gray)
# Bitwise-AND mask and original image
res2 = cv2.bitwise_and(img,img, mask= mask)


titles = ['Original Image', 'Selected Gray Values', 'Hue', 'Result']
images = [img, res, hue, res2]
for i in xrange(4):
    plt.subplot(2,2,i+1),plt.imshow(images[i],'gray')
    plt.title(titles[i])
    plt.xticks([]),plt.yticks([])
plt.show()

Answer 2

1) If all you need is to find the dominant color why not find the histograms of each color channel? Find the dominant channel then segment only that channel using otsu? For example if I threshold only the hue I can get nice results. K-means could be an overkill for this task:

import cv2
import numpy as np
import matplotlib.pylab as plt

## Simple Otsu over hue
six = cv2.imread('7zovC.jpg')

##convert to hsv
hsv = cv2.cvtColor(six, cv2.COLOR_BGR2HSV)
hue = hsv[:, :, 0]

binary_img = cv2.threshold(hue, 128, 255, cv2.THRESH_OTSU)

plt.figure()
plt.imshow(binary_img*255)
plt.show()

2) Why not use all channels for clustering instead of just hue? What you need is clustering -> color quantization this link should be useful. This is for opencv version > 3.0.0

Note for python 2.4.11, cv2.kmeans has a slightly difference interface and you could use this instead:

def color_quantize(img, K):
    Z = img.reshape((-1, 3))

    # convert to np.float32
    Z = np.float32(Z)

    # define criteria, number of clusters(K) and apply kmeans()
    criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
    ret, label, center = cv2.kmeans(Z, 2, criteria, 10, cv2.KMEANS_RANDOM_CENTERS)

    # Now convert back into uint8, and make original image
    center = np.uint8(center)
    res = center[label.flatten()]
    quantized_img = res.reshape((img.shape))

    label_img = label.reshape((img.shape[:2]))
    return label_img, quantized_img



six = cv2.imread('7zovC.jpg')


##convert to hsv
hsv = cv2.cvtColor(six, cv2.COLOR_BGR2HSV)

K = 2
label_img, six_q = color_quantize(hsv, K)



plt.figure()
plt.imshow(label_img)

plt.show()

My results for color quantization were not impressive.