How to detect object from video using SVM

Question

This is my code for training the dataset of for example vehicles , when it train fully , i want it to predict the data(vehicle) from video(.avi) , how to predict trained data from video and how to add that part in it ? , i want that when the vehicle is shown in the video it count it as 1 and cout that the object is detected and if second vehicle come it increment the count as 2

    IplImage *img2;
    cout<<"Vector quantization..."<<endl;
    collectclasscentroids();
    vector<Mat> descriptors = bowTrainer.getDescriptors();
    int count=0;
    for(vector<Mat>::iterator iter=descriptors.begin();iter!=descriptors.end();iter++)
    {
       count += iter->rows;
    }
    cout<<"Clustering "<<count<<" features"<<endl;
    //choosing cluster's centroids as dictionary's words
    Mat dictionary = bowTrainer.cluster();
    bowDE.setVocabulary(dictionary);
    cout<<"extracting histograms in the form of BOW for each image "<<endl;
    Mat labels(0, 1, CV_32FC1);
    Mat trainingData(0, dictionarySize, CV_32FC1);
    int k = 0;
    vector<KeyPoint> keypoint1;
    Mat bowDescriptor1;
    //extracting histogram in the form of bow for each image 
   for(j = 1; j <= 4; j++)
    for(i = 1; i <= 60; i++)
            {
              sprintf( ch,"%s%d%s%d%s","train/",j," (",i,").jpg");
              const char* imageName = ch;
              img2 = cvLoadImage(imageName, 0); 
              detector.detect(img2, keypoint1);
              bowDE.compute(img2, keypoint1, bowDescriptor1);
              trainingData.push_back(bowDescriptor1);
              labels.push_back((float) j);
             }
    //Setting up SVM parameters
    CvSVMParams params;
    params.kernel_type = CvSVM::RBF;
    params.svm_type = CvSVM::C_SVC;
    params.gamma = 0.50625000000000009;
    params.C = 312.50000000000000;
    params.term_crit = cvTermCriteria(CV_TERMCRIT_ITER, 100, 0.000001);
    CvSVM svm;



    printf("%s\n", "Training SVM classifier");

    bool res = svm.train(trainingData, labels, cv::Mat(), cv::Mat(), params);

    cout<<"Processing evaluation data..."<<endl;


    Mat groundTruth(0, 1, CV_32FC1);
    Mat evalData(0, dictionarySize, CV_32FC1);
    k = 0;
    vector<KeyPoint> keypoint2;
    Mat bowDescriptor2;


    Mat results(0, 1, CV_32FC1);;
    for(j = 1; j <= 4; j++)
      for(i = 1; i <= 60; i++)
         {
           sprintf( ch, "%s%d%s%d%s", "eval/", j, " (",i,").jpg");
           const char* imageName = ch;
           img2 = cvLoadImage(imageName,0);
           detector.detect(img2, keypoint2);
           bowDE.compute(img2, keypoint2, bowDescriptor2);
           evalData.push_back(bowDescriptor2);
           groundTruth.push_back((float) j);
           float response = svm.predict(bowDescriptor2);
           results.push_back(response);
         }



    //calculate the number of unmatched classes 
    double errorRate = (double) countNonZero(groundTruth- results) / evalData.rows;

The question isThis code is not predicting from video , i want to know how to predict it from the video , mean like i want to detect the vehicle from movie , like it should show 1 when it find the vehicle from movie

For those who didn't understand the question :

I want to play a movie in above code

VideoCapture cap("movie.avi"); //movie.avi is with deleted background

Suppose i have a trained data which contain vehicle's , and "movie.avi" contain 5 vehicles , so it should detect that vehicles from the movie.avi and give me 5 as output

How to do this part in the above code

Answer 1

From looking at your code setup

params.svm_type = CvSVM::C_SVC;

it appears that you train your classifier with more than two classes. A typical example in traffic scenario could be cars/pedestrians/bikes/... However, you were asking for a way to detect cars only. Without a description of your training data and your video it's hard to tell, if your idea makes sense. I guess what the previous answers are assuming is the following:

You loop through each frame and want to output the number of cars in that frame. Thus, a frame may contain multiple cars, say 5. If you take the whole frame as input to the classifier, it might respond "car", even if the setup might be a little off, conceptually. You cannot retrieve the number of cars reliably with this approach.

Instead, the suggestion is to try a sliding-window approach. This means, for example, you loop over each pixel of the frame and take the region around the pixel (called sub-window or region-of-interest) as input to the classifier. Assuming a fixed scale, the sub-window could have a size of 150x50px as well as your training data would. You might fixate the scale of the cars in your training data, but in real-world videos, the cars will be of different size. In order to find a car of different scale, let's say it's two-times as large as in the training data, the typical approach is to scale the image (say with a factor of 2) and repeat the sliding-window approach.

By repeating this for all relevant scales you end up with an algorithm that gives you for each pixel location and each scale the result of your classifier. This means you have three loops, or, in other words, there are three dimensions (image width, image height, scale). This is best understood as a three-dimensional pyramid. "Why a pyramid?" you might ask. Because each time the image is scaled (say 2) the image gets smaller (/larger) and the next scale is an image of different size (for eample half the size).

The pixel locations indicate the position of the car and the scale indicates the size of it. Now, if you have an N-class classifier, each slot in this pyramid will contain a number (1,...,N) indicating the class. If you had a binary classifier (car/no car), then you would end up with each slot containing 0 or 1. Even in this simple case, where you would be tempted to simply count the number of 1 and output the count as the number of cars, you still have the problem that there could be multiple responses for the same car. Thus, it would be better if you had a car detector that gives continous responses between 0 and 1 and then you could find maxima in this pyramid. Each maximum would indicate a single car. This kind of detection is successfully used with corner features, where you detect corners of interest in a so-called scale-space pyramid.

To summarize, no matter if you are simplifying the problem to a binary classification problem ("car"/"no car"), or if you are sticking to the more difficult task of distinguishing between multiple classes ("car"/"animal"/"pedestrian"/...), you still have the problem of scale and location in each frame to solve.

Answer 2

You need detector, not classifier. Take a look at Haar cascades, LBP cascades, latentSVM, as mentioned before or HOG detector.

I'll explain why. Detector usually scan image by sliding window, line by line. In several scales. In every window detector solve problem: "object/not object". It may give you rough results but very fast. Classifiers such as BOW, works very slow for this task. Then you should apply classifiers to regions, found by detector.

Answer 3

You should implement a sliding window approach. In each window, you should apply the SVM to get candidates. Then, once you've done it for the whole image, you should merge the candidates (if you detected an object, then it is very likely that you'll detect it again in shift of a few pixels - that's the meaning of candidates).

Take a look at the V&J code at openCV or the latentSVM code (detection by parts) to see how it's done there.

By the way, I would use the LatentSVM code (detection by parts) to detect vehicles. It has trained models for cars and for buses.

Good luck.

Answer 4

The code you have for using images is written with OpenCV's C interface so it's probably easy to stick with that rather than use the C++ video interface.

In which case somthing along these lines should work:

CvCapture *capture = cvCaptureFromFile("movie.avi");

IplImage *img = 0;
while(img = cvQueryFrame(capture))
{
       // Process image
       ...
}