How to identify holes connected by lines in binary graph

Question

I have picked up a code segement from the top voted author's answer from this question:

https://answers.opencv.org/question/9863/fill-holes-of-a-binary-image/

Refomatted it as:

cv::Mat image = cv::imread("image.jpg", 0);

cv::Mat image_thresh;
cv::threshold(image, image_thresh, 125, 255, cv::THRESH_BINARY);

// Loop through the border pixels and if they're black, floodFill from there
cv::Mat mask;
image_thresh.copyTo(mask);
for (int i = 0; i < mask.cols; i++) {
 if (mask.at<char>(0, i) == 0) {
     cv::floodFill(mask, cv::Point(i, 0), 255, 0, 10, 10);
 }   
  if (mask.at<char>(mask.rows-1, i) == 0) {

     cv::floodFill(mask, cv::Point(i, mask.rows-1), 255, 0, 10, 10);
   }
 }

 for (int i = 0; i < mask.rows; i++) {

  if (mask.at<char>(i, 0) == 0) {
    cv::floodFill(mask, cv::Point(0, i), 255, 0, 10, 10);
  }

  if (mask.at<char>(i, mask.cols-1) == 0) {
     cv::floodFill(mask, cv::Point(mask.cols-1, i), 255, 0, 10, 10);
  } 
}


 // Compare mask with original.
cv::Mat newImage;
image.copyTo(newImage);
for (int row = 0; row < mask.rows; ++row) {
 for (int col = 0; col < mask.cols; ++col) {
    if (mask.at<char>(row, col) == 0) {
        newImage.at<char>(row, col) = 255;
    }           
 }
}

cv::imshow("filled image", mask);
cv::imshow("Final image", newImage);
cv::imwrite("final.jpg", newImage);
cv::waitKey(0);

return 0;

I understand it used floodfill algorithm to try to fill holes, and I've tested on another sample image:

and it works really well by detecting all the 9 holes.

However, I tried another slighly complex image:

This time it won't work and it will fill the whole graph with white, and the number of holes it detects is 1700.

I think I might be lacking a siginificant amnout of morphological knowledge here, but I assume maybe I should do a "shirnking" on the failed image first, before insert it into the author's code?

Could experts share some thoughts with me because I could not find very similar graphs on google for hole detection. So what is the special about holes when two holes are connected with a white path in the binary image?Thanks in advance!

Answer 1

Another simple approach is to use contour area filtering. The idea is to find contours then perform contour area filtering with cv2.contourArea and a minium threshold area. If a contour passes this threshold filter then we count it as a valid contour. Here's the results with a threshold value for 500. You may need to change it depending on your image.

---

I implemented it in Python but you can easily adapt the same approach into C++

import cv2
import numpy as np

# Load image, grayscale, Otsu's threshold
image = cv2.imread('1.png')
mask = np.zeros(image.shape[:2], dtype=np.uint8)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]

# Find contours and filter using contour area
cnts = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
    area = cv2.contourArea(c)
    if area < 500:
        cv2.drawContours(image,[c],0,(36,255,12),-1)
        cv2.drawContours(mask,[c],0,255,1)

cv2.imshow('mask', mask)
cv2.imshow('image', image)
cv2.waitKey()

Answer 2

There is a problem in your image, it has a thin white bar surrounding 3 sides of your image. This bar is also connected to the 4 white rectangles on the left which creates an extra enclosed contour/level that confuses the 'floodfill' I guess.

I personally do not prefer to use the 'floodfill' method to solve the problem of finding holes within contours. I prefer to use the 'findcontour' method with the 'hierarchy' option. Please have a look at it here. At first glance, it might look a bit complicated but it gives all the information we need.

The holes you are looking for have two properties:

1. They are a child contour (a hole)
2. They have no other contour within them (not a parent)

The code for finding these holes is:

auto image = cv::imread(in_img_path, cv::ImreadModes::IMREAD_GRAYSCALE);
cv::threshold(image, image, 128, 255, cv::THRESH_OTSU);
std::vector<std::vector<cv::Point>> contours, selected_contours;
std::vector<cv::Vec4i> hierarchy;   
cv::findContours(image, contours, hierarchy, cv::RetrievalModes::RETR_TREE, cv::ContourApproximationModes::CHAIN_APPROX_SIMPLE);
for (int i = 0; i < contours.size(); i++) {
    if (hierarchy[i][2] == -1 && hierarchy[i][3] != -1) //the contour has no children but has a parent
        selected_contours.emplace_back(std::move(contours[i]));
}
cv::Mat drawing_image(image.size(), image.type(), cv::Scalar::all(0));

for (int i = 0; i < selected_contours.size(); i++) {
    cv::drawContours(drawing_image, selected_contours, i, cv::Scalar(255), 1);
}

Edit: I tried it, it seems that the first check is redundant in this case. The following condition is sufficient:

if (hierarchy[i][3] != -1) // the contour has a parent

The number of holes (size of selected_contours) is: 71

And the 'drawing_image' will look like this: