HSV color classification with opencv

Question

I'm trying to recognize four HSV intervals in a picture, the result images should be made of 0,1,2,3 where these stand for the color index that was recognized.

I'd like to do that using the opencv API as more as I can, in order to reduce execution time and loop overhead. I've tried using inRange and addWeighted but I didn't come up with a functional algorithm. Could you help me?

for example, let's suppose I want to classify

110<H<130 && S > 100 && V > 100 as color0
140<H<160 && S > 100 && V > 100 as color1
50<H<70 && S > 100 && V > 100 as color2

The problem is that I have to work on all the three channel simultaneously with different intervals, so I suppose that LUT way (1 channel or 3 different channels) isn't the correct way to get a 1D classification output matrix from a 3D input matrix.

Answer 1

I haven't tried it, but I imagine a Lookup Table (LUT) would be ideal. Say your ranges were 0-30, 31-78, 79-91 and 92-100, you would make a 256-element LUT where the first 31 entries were zero, the next 48 entries were one and so on. Then call cv::LUT().

The advantage of a LUT is that the output values are simply taken from a table indexed by the Hue value, which means there are not multiple if statements per pixel which means the CPU does not stall while determining the branch destination - so it should be very fast.

---

I did some experiments and could not get the built-in LUT() function to perform fast for this setup - also had to use split() which slowed things down. I used this HSL colour wheel as a test image.

Here is the code. First it uses if statements to process the conditions and times it, and then it repeats the processing but uses 3 LUTs to determine the output values:

// https://stackoverflow.com/q/49333257/2836621
#include <iostream>
#include <cstdio>
#include <chrono>
#include <thread>
#include <opencv2/opencv.hpp>

using namespace cv;
using namespace std;

int
main(int argc,char*argv[])
{
   // Open input and check success
   Mat img = cv::imread("start.png",-CV_LOAD_IMAGE_ANYDEPTH);
   if (img.empty()){
      cerr << "ERROR: Failed to open start.png" << endl;
      exit(1);
   }

   // Create HSV version
   Mat hsvMat;
   cv::cvtColor(img,hsvMat,CV_BGR2HSV);

   // Create result Mat, same dimensions as input, and pointer to its data
   Mat result;
   cv::cvtColor(img,result,CV_BGR2GRAY);
   unsigned char* r;

   // Various constants
   const int Sthreshold = 100;
   const int Vthreshold = 100;
   const int Hthreshold[]={110,130,140,160,50,70};

   // Process with if statements and time it
   double t = (double)getTickCount();

   // Iterate over rows and columns of hsvMat
   Vec3b *thisRow;
   for(int j=0;j<hsvMat.rows;++j)
   {
       // Get pointers to input and output rows
       thisRow = hsvMat.ptr<Vec3b>(j);
       r = result.ptr<unsigned char>(j);
       // Iterate over cols
       for(int i=0;i<hsvMat.cols;++i)
       {
          auto H=thisRow[i][0];  // Pick up this pixel's Hue
          auto S=thisRow[i][1];  // Pick up this pixel's Sat
          auto V=thisRow[i][2];  // Pick up this pixel's Value
          // Pre-set output colour to black
          unsigned char v=0;
          if((V>Vthreshold) && (S>Sthreshold)){
             // Set result based on Hue
             if((H>Hthreshold[0]) && (H<Hthreshold[1])){
                v=64;
             } else if((H>Hthreshold[2]) && (H<Hthreshold[3])){
                v=128;
             } else if((H>Hthreshold[4]) && (H<Hthreshold[5])){
                v=192;
             }
          }
          r[i]=v;
       }
   }

   t = (((double)getTickCount() - t)*1000000)/getTickFrequency();
   cout << "Time with if statements: " << t << "us" << std::endl;

   // Write to disk for checking
   imwrite("resultptr.png",result);

   // Now do setup for LUT method

   // Pre-create LUTs for Hue, Saturation, Value
   unsigned char HLUT[256]={0};
   unsigned char SLUT[256]={0};
   unsigned char VLUT[256]={0};
   for(int i=0;i<256;i++){
      if(i>Sthreshold){SLUT[i]=1;}
      if(i>Vthreshold){VLUT[i]=1;}
      if((i>Hthreshold[0]) && (i<Hthreshold[1])){
         HLUT[i]=64;
      } else if((i>Hthreshold[2]) && (i<Hthreshold[3])){
         HLUT[i]=128;
      } else if((i>Hthreshold[4]) && (i<Hthreshold[5])){
         HLUT[i]=192;
      }
   }

   // Process with LUT and time it
   t = (double)getTickCount();

   // Iterate over rows and columns of hsvMat
   for(int j=0;j<hsvMat.rows;++j)
   {
       // Get pointers to input and output rows
       thisRow = hsvMat.ptr<Vec3b>(j);
       r = result.ptr<unsigned char>(j);
       // Iterate over cols
       for(int i=0;i<hsvMat.cols;++i)
       {
          auto H=thisRow[i][0];              // Pick up this pixel's Hue
          auto S=thisRow[i][1];              // Pick up this pixel's Sat
          auto V=thisRow[i][2];              // Pick up this pixel's Value
          r[i]=HLUT[H] * SLUT[S] * VLUT[V];  // Lookup output pixel
       }
   }

   t = (((double)getTickCount() - t)*1000000)/getTickFrequency();
   cout << "Time with LUT: " << t << "us" << std::endl;

   // Write to disk for checking
   imwrite("resultLUT.png",result);
}

The result is:

And the timings on my iMac (with clang++ -march=native -O3) are:

Time with if statements: 38.429us
Time with LUT: 24.31us

If anyone wants to take the code and try alternative methods for the second set of loops, please feel free to try and publish what you get, but keep the first set of loops unchanged for reference.