Blob Detection with light-colored blobs

Question

I am having some issues with detecting specific "blobs" in a set of images. Not all images are the same, but I suppose the same parameters would be used to detect anyways.

If you zoom in, you will see small, yellow aphids on the leaf. My goal is to single these out and count them. I don't really need to do much to the image, just obtain a count of them.

Right now, I have this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Emgu.CV;
using Emgu.CV.Features2D;
using Emgu.CV.Structure;
using Emgu.CV.Util;

namespace AphidCounter
{
    class Program
    {
        static void Main(string[] args)
        {
            // Read image
            Mat im_in = CvInvoke.Imread("myimage1.jpg", Emgu.CV.CvEnum.LoadImageType.Grayscale);
            //Mat im_in = CvInvoke.Imread("myimage2.png", Emgu.CV.CvEnum.LoadImageType.Color);
            Mat im = im_in;
            CvInvoke.Threshold(im_in, im, 40, 255, Emgu.CV.CvEnum.ThresholdType.BinaryInv);  // 60, 255, 1

            //CvInvoke.NamedWindow("Blob Detector", Emgu.CV.CvEnum.NamedWindowType.AutoSize);

            DetectBlobs(im, 0);

            CvInvoke.WaitKey(0);
        }

        static void DetectBlobs(Mat im, int c)
        {

            int maxT = 50;
            int minA = 125; // Minimum area in pixels
            int maxA = 550; // Maximum area in pixels

            SimpleBlobDetectorParams EMparams = new SimpleBlobDetectorParams();
            SimpleBlobDetector detector;

            EMparams.MinThreshold = 0;
            EMparams.MaxThreshold = 100;

            if (minA < 1) minA = 1;
            EMparams.FilterByArea = true;
            EMparams.MinArea = minA;
            EMparams.MaxArea = maxA;

            if (maxT < 1) maxT = 1;
            EMparams.MinConvexity = (float)maxT / 1000.0F; // 0.67

            EMparams.FilterByInertia = true;
            EMparams.MinInertiaRatio = 0.01F;

            EMparams.FilterByColor = true;
            EMparams.blobColor = 0;

            VectorOfKeyPoint keyPoints = new VectorOfKeyPoint();

            detector = new SimpleBlobDetector(EMparams);
            detector.DetectRaw(im, keyPoints);

            Mat im_with_keypoints = new Mat();
            Bgr color = new Bgr(0, 0, 255);
            Features2DToolbox.DrawKeypoints(im, keyPoints, im_with_keypoints, color, Features2DToolbox.KeypointDrawType.DrawRichKeypoints);

            // Show blobs
            CvInvoke.Imwrite("keypoints1.jpg", im_with_keypoints);
            CvInvoke.Imshow("Blob Detector " + keyPoints.Size, im_with_keypoints);

            System.Console.WriteLine("Number of keypoints: " + keyPoints.Size);

        }
    }
}

However, this is the result:

Am I not getting the parameters right? Or is there something else that I'm missing?

Answer 1

It is not because of some wrong parameters. The image segmentation part itself has its limitation.

Grayscale based thresholding may not work when the contrast between the blob and the background is very low. Yet a threshold value around 160 is quite tolerable in this example but not any accurate.

I would suggest to go for colour based thresholding since there is a decent colour gap.

Here is a C++ implementation of colour based thresholding. Blobs are filtered using the same SimpleBlobDetector.

I have converted the image from RGB to ‘Lab’ for better segmentation.

As the image provided is too huge, it took more time to process. So I cropped a key part of the image and tuned the blob params for the same. So I provide the cropped image too (755 x 494px).

Colour based thresholding and blob filtering:

#include "opencv2\imgproc\imgproc.hpp";
#include "opencv2\highgui\highgui.hpp";
#include "opencv2\features2d\features2d.hpp";

using namespace cv;
using namespace std;

void main()
{
    char image_path[] = "E:/Coding/media/images/leaf_small.jpg";
    Mat img_color, img_lab, img_thresh, img_open, img_close, img_keypoints;

    img_color = imread(image_path, IMREAD_ANYCOLOR);

    //Convert image to CIE Lab colorspace for better colour based segmentation
    cvtColor(img_color, img_lab, CV_BGR2Lab);

    //create window before creating trackbar
    namedWindow("win_thresh", WINDOW_NORMAL);
    namedWindow("win_blob", WINDOW_NORMAL);

    //Using trackbar calculate the range of L,a,b values to seperate blobs
    int low_L = 150, low_A = 0, low_B = 155,
        high_L = 255, high_A = 255, high_B = 255;

    //*Use trackbars to caliberate colour thresholding
    createTrackbar("low_L", "win_thresh", &low_L, 255);
    createTrackbar("low_A", "win_thresh", &low_A, 255);
    createTrackbar("low_B", "win_thresh", &low_B, 255);
    createTrackbar("high_L", "win_thresh", &high_L, 255);
    createTrackbar("high_A", "win_thresh", &high_A, 255);
    createTrackbar("high_B", "win_thresh", &high_B, 255);

    int minArea = 35, maxArea = 172, minCircularity = 58, minConvexity = 87, minInertiaRatio = 21;

    //Use trackbar and set Blob detector parameters
    createTrackbar("minArea", "win_blob", &minArea, 200);
    createTrackbar("maxArea", "win_blob", &maxArea, 200);
    createTrackbar("minCircular", "win_blob", &minCircularity, 99);
    createTrackbar("minConvex", "win_blob", &minConvexity, 99);
    createTrackbar("minInertia", "win_blob", &minInertiaRatio, 99);

    SimpleBlobDetector::Params params;
    vector<KeyPoint> keypoints;

    while (waitKey(1) != 27) //press 'esc' to quit
    {
        //inRange thresholds basedon the Scalar boundaries provided
        inRange(img_lab, Scalar(low_L, low_A, low_B), Scalar(high_L, high_A, high_B), img_thresh);

        //Morphological filling
        Mat strucElement = getStructuringElement(CV_SHAPE_ELLIPSE, Size(5, 5), Point(2, 2));
        morphologyEx(img_thresh, img_close, MORPH_CLOSE, strucElement);

        imshow("win_thresh", img_close);

        //**SimpleBlobDetector works only in inverted binary images
        //i.e.blobs should be in black and background in white.
        bitwise_not(img_close, img_close); // inverts matrix

        //Code crashes if minArea or any miin value is set to zero
        //since trackbar starts from 0, it is adjusted here by adding 1
        params.filterByArea = true;
        params.minArea = minArea + 1;
        params.maxArea = maxArea + 1;

        params.filterByCircularity = true;
        params.filterByConvexity = true;
        params.filterByInertia = true;

        params.minCircularity = (minCircularity + 1) / 100.0;
        params.minConvexity = (minConvexity + 1) / 100.0;
        params.minInertiaRatio = (minInertiaRatio + 1) / 100.0;

        SimpleBlobDetector detector(params);
        detector.detect(img_close, keypoints);
        drawKeypoints(img_color, keypoints, img_keypoints, Scalar(0, 0, 255), DrawMatchesFlags::DEFAULT);

        stringstream displayText;
        displayText = stringstream();
        displayText << "Blob_count: " << keypoints.size();
        putText(img_keypoints, displayText.str(), Point(0, 50), CV_FONT_HERSHEY_PLAIN, 2, Scalar(0, 0, 255), 2);

        imshow("win_blob", img_keypoints);
    }
    return;
}

Output Screenshot

Tune the blob parameters according to the actual HD image.

Since the veins of the leaf are almost of the same colour and intensity of the aphid, this method also may utterly fail when an aphid sits close to or exactly on top of a vein.

This can be an ad-hoc fix but not robust enough. There got to be a simple and robust method to achieve the result, using some filters, transformation or edge detection. Please share any other optimal solution if available.

EDIT: Opting Grayscale thresholding as previous approach failed

Colour thresholding approach failed for this_image

Colour based thresholding has a very narrow bandwidth, if the image falls within the bandwidth the accuracy will be really good, on the other hand colour shifts totally ruin the accuracy. Since you will be processing 100s of images, colour thresholding may not be suitable.

I tried normal Grayscale thresholding with some morphological erosion and filling, and got a decent accuracy. Also Grayscale thresholding has better immunity to colour shifts.

Additionally we have auto thrsholding option using OTSU Thresholding which selects the threshold value based on the image.

Code snippet:

threshold(img_gray, img_thresh, 0, 255, THRESH_OTSU);

Mat strucElement = getStructuringElement(CV_SHAPE_ELLIPSE, Size(3, 3), Point(1, 1));
morphologyEx(img_thresh, img_open, MORPH_OPEN, strucElement);

Rest of the code remains the same.

Parameter values:

minArea = 75, maxArea = 1000, minCircularity = 50, minConvexity = 20, minInertiaRatio = 15

The white ants are hard to differentiate from aphids as we are not using colour information. So the min_area has to be carefully tuned in order to exclude them.

Processed images can be found here img_1, img_2.

Tweak the morphology methods and blob parameters to obtain an optimal average count.