Opencv - How to calculate moving object distance from camera and speed in Haar Classifier Cascade?

Question

I'm working on Opencv Project, I'm using C++. I am stuck at computing moving objects distance and speed.

---

Problem:

I have detected moving cars in video using Haar classifier in Opencv.
But didn't find any way to calculate moving objects
distance from camera as well as moving objects speed. 
I want to achieve this using only 1 Camera. I am not using Stereo vision. 

Here is my code:

#include <stdio.h>
#include <opencv/cv.h>
#include <opencv/highgui.h>

CvHaarClassifierCascade *cascade;
CvMemStorage            *storage;

void detect(IplImage *img);

int main(int argc, char** argv)
{
  CvCapture *capture;
  IplImage  *frame;
  int input_resize_percent = 100;

  if(argc < 3)
  {
    std::cout << "Usage " << argv[0] << " cascade.xml video.avi" << std::endl;
    return 0;
  }

  if(argc == 4)
  {
    input_resize_percent = atoi(argv[3]);
    std::cout << "Resizing to: " << input_resize_percent << "%" << std::endl;
  }

  cascade = (CvHaarClassifierCascade*) cvLoad(argv[1], 0, 0, 0);
  storage = cvCreateMemStorage(0);
  capture = cvCaptureFromAVI(argv[2]);

  assert(cascade && storage && capture);

  cvNamedWindow("video", 1);

  IplImage* frame1 = cvQueryFrame(capture);
  frame = cvCreateImage(cvSize((int)((frame1->width*input_resize_percent)/100) , (int)((frame1->height*input_resize_percent)/100)), frame1->depth, frame1->nChannels);

  const int KEY_SPACE  = 32;
  const int KEY_ESC    = 27;

  int key = 0;
  do
  {
    frame1 = cvQueryFrame(capture);

    if(!frame1)
      break;

    cvResize(frame1, frame);

    detect(frame);

    key = cvWaitKey(10);

    if(key == KEY_SPACE)
      key = cvWaitKey(0);

    if(key == KEY_ESC)
      break;

  }while(1);

  cvDestroyAllWindows();
  cvReleaseImage(&frame);
  cvReleaseCapture(&capture);
  cvReleaseHaarClassifierCascade(&cascade);
  cvReleaseMemStorage(&storage);

  return 0;
}

void detect(IplImage *img)
{
  CvSize img_size = cvGetSize(img);
  CvSeq *object = cvHaarDetectObjects(
    img,
    cascade,
    storage,
    1.1, //1.1,//1.5, //-------------------SCALE FACTOR
    1, //2        //------------------MIN NEIGHBOURS
    0, //CV_HAAR_DO_CANNY_PRUNING
    cvSize(0,0),//cvSize( 30,30), // ------MINSIZE
    img_size //cvSize(70,70)//cvSize(640,480)  //---------MAXSIZE
    );

  std::cout << "Total: " << object->total << " cars" << std::endl;
  for(int i = 0 ; i < ( object ? object->total : 0 ) ; i++)
  {
    CvRect *r = (CvRect*)cvGetSeqElem(object, i);
    cvRectangle(img,
      cvPoint(r->x, r->y),
      cvPoint(r->x + r->width, r->y + r->height),
      CV_RGB(255, 0, 0), 2, 8, 0);
  }

  cvShowImage("video", img);
}

If you have any example please provide for better understanding. Its appreciated.

Thanks

Answer 1

use this code I have done it in android

    static constexpr int32_t cam_number     = 1;                 /**< The number of the camera, the 0 is the built in my computer. */
    static constexpr int32_t cam_width      = 640;               /**< Width of the video's resolution. */
    static constexpr int32_t cam_height     = 480;               /**< Height of the video's resolution. */
    static constexpr int32_t threshold_min  = 245;               /**< Minimum value of the binary threshold. */
    static constexpr int32_t threshold_max  = 255;               /**< Maximum value of the binary threshold. */

Look-up table for linear interpolation. If you want to make your own version, you have to re-measure these values.

static std::vector<double> pixel  = {42.0, 94.0, 122.0, 139.0, 150.0, 157.0, 163.0, 168.0, 171.0};  /**< Measured values of pixels. */
static std::vector<double> cm     = {20.0, 30.0, 40.0,  50.0,  60.0,  70.0,  80.0,  90.0,  100.0};  /**< Measured values of centimeters. */

/* Initialize the video formats. */
cv::Mat video=srcImg;
cv::Mat video_gray=resultImage;
cv::Mat video_black_white;
    try {
        /* Vectors for contours and hierarchy. */
        std::vector<std::vector<cv::Point>> contours;
        std::vector<cv::Vec4i> hierarchy;

        /* Get a new frame from the camera, convert it to grayscale, then make into black&white with binary threshold. */

        cv::cvtColor(video, video_gray, cv::COLOR_RGB2GRAY);
        cv::threshold(video_gray, video_black_white, threshold_min, threshold_max,
                      cv::THRESH_BINARY);

        /* Get contours with full hierararchy and simple approximation. */
        cv::findContours(video_black_white, contours, hierarchy, cv::RETR_TREE,
                         cv::CHAIN_APPROX_SIMPLE, cv::Point(0, 0));

        /* If there are no contours, skip everything, otherwise there would be an exception. */
        if (contours.size()) {
            /* Get moments. */
            cv::Moments m = cv::moments(contours[0]);
            /* Protection from divison by zero. */
            if (m.m00 > 0.0) {
                /* Calculate the x,y coordinates of the laser point. */
                double coord_x = m.m10 / m.m00;
                double coord_y = m.m01 / m.m00;

                /* Make sure, that we are in the look-up table's range. */
                if ((coord_y > pixel[0]) && (coord_y < pixel[pixel.size() - 1])) {
                    /* Find the place of the coordinate in the look-up table. */
                    uint32_t i = 0;
                    while (coord_y > pixel[i + 1]) {
                        i++;
                    }
                    /* Calculate the value with linear interpolation. */
                    double distance = cm[i] + ((coord_y - pixel[i]) * (cm[i + 1] - cm[i]) /
                                               (pixel[i + 1] - pixel[i]));
                    dscmm=distance;


     std::cout << "X: " << coord_x << "\tY: " << coord_y << "\tDistance: "
//                                  << distance << "\n";


                    /* Draw a circle on the laser and put a text with the distance on it. */
                    cv::circle(video, cv::Point(coord_x, coord_y), 5, cv::Scalar(0, 0, 0), 1,
                               8);
                    cv::putText(video, std::to_string(distance), cv::Point(coord_x, coord_y),
                                cv::FONT_HERSHEY_SCRIPT_SIMPLEX, 0.5, cv::Scalar(255, 0, 0), 1);
                }
            }
        }
        /* Show the picture. */

        /* Press any key to exit. */

    }
        /* Write out if there is an error. */
    catch (std::exception &e) {
        LOGD( "JNI:", e.what());
    }

Answer 2

I doubt about accuracy, but below mentioned way can help you to find distance of object(moving) to some extent.

• Steps:
• Find the suitable network to feed your frames for identifying the objects in the video.
• Get coordinates of the detections and probably we could find length/area of the detection in further.
• Before doing all this, find the focal length of the camera with an object of known size placed at known distance.(Refer Adrian pyimagesearch website).
• With known focal length, and the moving object of known size, we can find the distance of the object while moving with the change in area of detected frame.