Background removal from images with OpenCV in Android

Question

I want to remove image background with Open CV in Android. Code is working fine but output quality not as per expectation. I followed java documentation for code reference:

https://opencv-java-tutorials.readthedocs.io/en/latest/07-image-segmentation.html

Thanks

original Image

My output Expected output

My code snippet in Android:

private fun doBackgroundRemoval(frame: Mat): Mat? {
    // init
    val hsvImg = Mat()
    val hsvPlanes: List = ArrayList()
    val thresholdImg = Mat()
    var thresh_type = Imgproc.THRESH_BINARY_INV
    thresh_type = Imgproc.THRESH_BINARY

    // threshold the image with the average hue value
    hsvImg.create(frame.size(), CvType.CV_8U)
    Imgproc.cvtColor(frame, hsvImg, Imgproc.COLOR_BGR2HSV)
    Core.split(hsvImg, hsvPlanes)

    // get the average hue value of the image
    val threshValue: Double = getHistAverage(hsvImg, hsvPlanes[0])
    threshold(hsvPlanes[0], thresholdImg, threshValue, 78.0, thresh_type)
    Imgproc.blur(thresholdImg, thresholdImg, Size(1.toDouble(), 1.toDouble()))

    val kernel1 =
        Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, Size(11.toDouble(), 11.toDouble()))
    val kernel2 = Mat.ones(3, 3, CvType.CV_8U)
    // dilate to fill gaps, erode to smooth edges
    Imgproc.dilate(thresholdImg, thresholdImg, kernel1, Point(-1.toDouble(), -1.toDouble()), 1)
    Imgproc.erode(thresholdImg, thresholdImg, kernel2, Point(-1.toDouble(), -1.toDouble()), 7)
    threshold(thresholdImg, thresholdImg, threshValue, 255.0, Imgproc.THRESH_BINARY_INV)

    // create the new image
    val foreground = Mat(
        frame.size(), CvType.CV_8UC3, Scalar(
            255.toDouble(),
            255.toDouble(),
            255.toDouble()
        )
    )
    frame.copyTo(foreground, thresholdImg)
    val img_bitmap =
        Bitmap.createBitmap(foreground!!.cols(), foreground!!.rows(), Bitmap.Config.ARGB_8888)
    Utils.matToBitmap(foreground!!, img_bitmap)
    imageView.setImageBitmap(img_bitmap)

    return foreground
}

Red · Accepted Answer

The Code

import cv2
import numpy as np

def process(img):
    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    img_canny = cv2.Canny(img_gray, 10, 20)
    kernel = np.ones((13, 13))
    img_dilate = cv2.dilate(img_canny, kernel, iterations=1)
    return cv2.erode(img_dilate, kernel, iterations=1)
    
def get_mask(img):
    contours, _ = cv2.findContours(process(img), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
    blank = np.zeros(img.shape[:2]).astype('uint8')
    for cnt in contours:
        if cv2.contourArea(cnt) > 500:
            peri = cv2.arcLength(cnt, True)
            approx = cv2.approxPolyDP(cnt, peri * 0.004, True)
            cv2.drawContours(blank, [approx], -1, 255, -1) 
    return blank

img = cv2.imread("crystal.jpg")
img_masked = cv2.bitwise_and(img, img, mask=get_mask(img))

cv2.imshow("Masked", img_masked)
cv2.waitKey(0)

The Output

The Explanation

Import the necessary libraries:

import cv2
import numpy as np

Define a function to process an image to be fit for proper contour detection. In the function, first convert the image to grayscale, and then detect its edges using the canny edge detector. With the edges detected, we can dilate and erode them once to give the edges more body:

def process(img):
    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    img_canny = cv2.Canny(img_gray, 10, 20)
    kernel = np.ones((13, 13))
    img_dilate = cv2.dilate(img_canny, kernel, iterations=1)
    return cv2.erode(img_dilate, kernel, iterations=1)

Define a function to generate a mask for the image. After finding the contours of the image, define a grayscale blank image with the shape of the image, and draw every contour (of area greater than 400 to filter out noise) filled in onto the blank image. I also approximated the contours to smoothen things out a bit:

def get_mask(img):
    contours, _ = cv2.findContours(process(img), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
    blank = np.zeros(img.shape[:2]).astype('uint8')
    for cnt in contours:
        if cv2.contourArea(cnt) > 500:
            peri = cv2.arcLength(cnt, True)
            approx = cv2.approxPolyDP(cnt, peri * 0.004, True)
            cv2.drawContours(blank, [approx], -1, 255, -1) 
    return blank

Finally, read in the image, and mask the image using the cv2.bitwise_and method, along with the get_mask function we defined, which uses the process function we defined. Show the masked image in the end:

img = cv2.imread("crystal.jpg")
img_masked = cv2.bitwise_and(img, img, mask=get_mask(img))

cv2.imshow("Masked", img_masked)
cv2.waitKey(0)

Transparent Background

Instead of the cv2.bitwise_and method, you can use the cv2.merge method:

img = cv2.imread("crystal.jpg")
img_masked = cv2.merge(cv2.split(img) + [get_mask(img)])
cv2.imwrite("masked_crystal.png", img_masked)

Resulting image (screenshot):

Explanation:

Keeping in mind we already imported the cv2 module and the numpy module as np. We also defined a process function and a get_mask function, we can read in the image:

img = cv2.imread("crystal.jpg")

The cv2.split method takes in an image array and returns a list of every individual channel present in the image. In our case, we only have 3 channels, and in order to make the image transparent, we need a forth channel: the alpha channel. The cv2.merge method does the opposite of cv2.split; it takes in a list of individual channels and returns an image array with the channels. So next we get the bgr channels of the image in a list, and concatenate the mask of the image as the alpha channel:

img_masked = cv2.merge(cv2.split(img) + [get_mask(img)])

Lastly we can write the four channel image into a file:

cv2.imwrite("masked_crystal.png", img_masked)

Here are some more example of the cv2.merge method: Python cv2.merge() Examples

Background removal from images with OpenCV in Android

Answers (2)

The Code

The Output

The Explanation

Transparent Background

Related Questions