How to find the distance(physical in m and pixels) between two object(points) in image in OpenCV and C+++?

Question

Im using OpenCV and C++. I have already done the code where can find the center of the object. I also have the prior information about the distance between the camera and object(s) in the image. I need to compute the distance (real physical one in m or cm, and also the one in pixels) between two objects in the image, or between the two centers of those objects).

Here is the code to find the center moment of rectangle object. Similar approach would be to find the center of other shape (object).

int main(int argc,char** argv)
{

           Mat image = imread("000167.png");
           Mat gray,bw,dil,erd, dst_final;
       Mat new_src=image.clone();

     for(int y = 0; y < image.rows; y++ )
    {
       for(int x = 0; x < image.cols; x++ )
       {
              for(int c = 0; c < 3; c++ )
              {
                     new_src.at<Vec3b>(y,x)[c]= saturate_cast<uchar>( 1.5*(image.at<Vec3b>(y,x)[c] ));

              }
       }
     }


      cv::GaussianBlur(new_src, src_gray, cv::Size(3,3),1,1,BORDER_DEFAULT); //original
      medianBlur(new_src, src_gray, 11);
      blur( new_src, src_gray, Size(3,3) );
      cvtColor(src_gray,gray,CV_BGR2GRAY);
      Canny(gray,bw,600,1200,5,true);
                    Mat grad,bw1;
                    Mat morphKernel = getStructuringElement(MORPH_ELLIPSE, Size(20,10));
                    morphologyEx(bw, grad, MORPH_GRADIENT, morphKernel);        
                    threshold(grad, bw1, 255.0, 255.0, THRESH_BINARY | THRESH_OTSU);                

          vector<vector<Point> > contours;
      vector<vector<Point> > rough;
      vector<vector<Point> >rough_color;
      vector<vector<Point> >precise;
          vector<Vec4i> hierarchy;
      Mat dst = image.clone();
          findContours(bw1.clone(), contours, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);

      vector<Point> approx;

      for( int i = 0; i< contours.size(); i++ )
      {

           approxPolyDP(Mat(contours[i]), approx, arcLength(Mat(contours[i]), true) * 0.01, true);
           if (fabs(contourArea(approx)) <2000 || fabs(contourArea(approx))>50000)                                     
               continue;

                  Rect r = boundingRect(contours[i]);
                  if( (float) (r.height/r.width) > 1.5 && (float) (r.height/r.width) <3)
                  {
                   rough.push_back(approx);
                   }
      }

      for(int i = 0; i < rough.size(); i++)
      {
          Rect bound=boundingRect(rough[i]);
          Rect x, y, w, h = boundingRect(rough[i]);
          rectangle(dst,Point(bound.br().x,bound.br().y), Point(bound.tl().x,bound.tl().y),Scalar(255, 0, 0),2);
          RotatedRect rectPoint = minAreaRect(rough[i]);

              Point2f fourPoint2f_rough[4];
              rectPoint.points(fourPoint2f_rough);
          vector<Point> fourPoint_rough;
          for(int i = 0; i <4; i++)
         {
             fourPoint_rough.push_back(fourPoint2f_rough[i]);
         }


        {
            line(dst, fourPoint2f_rough[i], fourPoint2f_rough[i + 1], Scalar(255,0,0), 3);
        }
        line(dst, fourPoint2f_rough[0], fourPoint2f_rough[3], Scalar(255,0,0), 3); */
      }

      if(rough.size() !=0 )
      {
      for( int i = 0; i< rough.size(); i++ )
      {
          vector<Moments> mu(1);
          vector<Point2f> mc(1);
          int gray_level;
          // compute the central momment
             mu[0] = moments( rough[i], false );
             mc[0] = Point2f( mu[0].m10/mu[0].m00 , mu[0].m01/mu[0].m00 );
          circle( dst, mc[0], 4, Scalar(0,0,255), -1, 8, 0 );
          gray_level=gray.at<uchar>(mc[0]);
          if(gray_level<200 && gray_level>20)
          {rough_color.push_back(rough[i]);}
      }


      for(int i = 0; i < rough_color.size(); i++)
      {
          Rect bound=boundingRect(rough_color[i]);
          Rect x, y, w, h = boundingRect(rough_color[i]);
          RotatedRect rectPoint = minAreaRect(rough_color[i]);
              Point2f fourPoint2f_color[4];
              rectPoint.points(fourPoint2f_color);
         vector<Point> fourPoint_color;
         for(int i = 0; i <4; i++)
         {
             fourPoint_color.push_back(fourPoint2f_color[i]);
         }


         for (int i = 0; i < 3; i++)
        {
            line(dst, fourPoint2f_color[i], fourPoint2f_color[i + 1], Scalar(0,255,0), 3);
        }
        line(dst, fourPoint2f_color[0], fourPoint2f_color[3], Scalar(0,255,0), 3);
      }

      }
...

So just need a code to get the distance (real in cm or m and the one in pixels) between those two center moments(two centers of the objects in the image). I include the outcome image of the center on the rectangle window (red dot in the window). So in this case I would like to compute the distance between the window and the "OPEN" box sign.

Answer 1

This answer is about getting the metric distance between two points.

If:

• the two points are on a (geometrical) plane that is perpendicular to the optical axis of the camera
• you have the distance between the camera center and the plane

then the metric distance between these two points can be computed by using the projection basics:

• d_m: metric distance between the two points (m.)
• d_p: pixel distance between the two points
• f: focal (m.)
• k: nbe of pixels per meter of the camera sensor (=1/ps , ps:pixel size, ususally 5 to 10 um)
• z_M: distance between the two points (m.)

We have: d_p = k . f . d_M/Z_M

so: d_M = d_p . z_M / (k.f)

If the above conditions are not perfectly met, it will induce some error. It is unclear from the question if these requirements are fulfilled. If they are not, then it could be impossible to get an answer. Unless maybe the angle between the plane and the axis is known, in which case some basic trigonometry calculation will be required.