CODEWITHSUNDEEP
c++opencvcomputer-visionopticalflow
Pieter
Pieter

Reputation: 32755

cv::goodFeaturesToTrack doesn't return any features

I'm trying to use cv::calcOpticalFlowPyrLK but sometimes an internal assertion in that function fails. The assertion is npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0. I'm using OpenCV 2.3.1. The source code for this function is available here.

It's hard to wrap my head around their code, especially because of my limited experience with computer graphics and their lack of comments. Why is this assertion being triggered and what does it say about my problem?

Edit: I call the function as follows:

cv::calcOpticalFlowPyrLK(curBwFrame, prvFrame, features, newFeatures, trackingStatus, errors);

I found out that the features vector, which was obtained by calling cv::goodFeaturesToTrack(curBwFrame, features, 5, 0.2, 0.5, skinMask); with a non-empty mask that appears to be sufficiently big and a valid image, doesn't contain any features. How can this happen?

curBwFrame

curBwFrame

skinMask

skinMask

I'm able to reproduce the problem using the following code snippet:

#include <vector>
#include <cassert>
#include <opencv2\opencv.hpp>
using std::vector;
using namespace cv;

int main() {
    vector<Point2f> features;
    cv::Mat curBwFrame = imread("curBwFrame.png");
    cv::cvtColor(curBwFrame, curBwFrame, CV_RGB2GRAY);
    imwrite("test.png", curBwFrame);

    cv::Mat skinMask = imread("skinMask.png");
    cv::cvtColor(skinMask, skinMask, CV_RGB2GRAY);
    imwrite("test.png", skinMask);

    cv::goodFeaturesToTrack(curBwFrame, features, 5, 0.2, 0.5, skinMask);
    assert(features.size() > 0);

    return 0;
}

Upvotes: 6

Views: 14285

Answers (4)

Fredo
Fredo

Reputation: 63

If your code is similar to that:

 Imgproc.goodFeaturesToTrack(mCurrentFrame, initial, NUMBER_OF_FEATURES, 0.1, 10); 
 //The OpenCV opticalFlow will crash if the feature vector does not include any elements
 if (initial.elemSize() == 0) {
     return;
 }
 initial.convertTo(mPrevPts, CvType.CV_32FC2);

 Video.calcOpticalFlowPyrLK(mPreviousFrame, mCurrentFrame, mPrevPts, mNextPts, status,err);

Make sure you do not forget the statement checking the size of the feature vector (initial.elemSize() == 0).

If this vector is empty, the conversion from a Matrix of point to floating points done by the initial.convertTo() line will not happen and the assertion will show up when calling calOpticalFlowPyrLK

Upvotes: 0

Bobbi Bennett
Bobbi Bennett

Reputation: 1646

Does the image start out in color? Use cv::transform to enhance the color contrast before converting to grey. Shoot for a full range of grey, from 0 to 255. Don't worry about saturating the image outside the mask.

Upvotes: 0

Ian Medeiros
Ian Medeiros

Reputation: 1776

The main problem are your parameters.In the OpenCV 2.3.2 documentation (no compatibility change between 2.3.1) this is the description of the method parameters:

void goodFeaturesToTrack(InputArray image, OutputArray corners, int maxCorners, double qualityLevel, double minDistance, InputArray mask=noArray(), int blockSize=3, bool useHarrisDetector=false, double k=0.04 )

Parameters:

  • image – Input 8-bit or floating-point 32-bit, single-channel image.
  • corners – Output vector of detected corners.
  • maxCorners – Maximum number of corners to return. If there are more corners than are found, the strongest of them is returned.
  • qualityLevel – Parameter characterizing the minimal accepted quality of image corners. The parameter value is multiplied by the best corner quality measure, which is the minimal eigenvalue (see cornerMinEigenVal() ) or the Harris function response (see cornerHarris() ). The corners with the quality measure less than the product are rejected. For example, if the best corner has the quality measure = 1500, and the qualityLevel=0.01 , then all the corners with the quality measure less than 15 are rejected.
  • minDistance – Minimum possible Euclidean distance between the returned corners.
  • mask – Optional region of interest. If the image is not empty (it needs to have the type CV_8UC1 and the same size as image ), it specifies the region in which the corners are detected.
  • blockSize – Size of an average block for computing a derivative covariation matrix over each pixel neighborhood. See cornerEigenValsAndVecs() .
  • useHarrisDetector – Parameter indicating whether to use a Harris detector (see cornerHarris()) or cornerMinEigenVal().
  • k – Free parameter of the Harris detector.

I recommend you to play a little with qualityLevel and minDistance to suffice your needs.

Upvotes: 5

Rui Marques
Rui Marques

Reputation: 8904

Have you tried goodFeaturesToTrack without a mask to see if it detects features inside the masked region? It is possible that, because the image is dark, and the region is a bit textureless, that goodFeaturesToTrack fails to find features there.

You might also try ORB or FAST instead of goodFeaturesToTrack. I have successfully used ORB with calcOpticalFlowPyrLK (but didn't try to use a mask).

Or you could try to brighten up the image or even enhance the contrast. Not really sure if this brings improvements because I think the biggest problem of your scenario is that objects in the scene do not have enough texture or corners, which are the more suitable features for these detectors. I recommend that you try ORB and see if you get more points.

Upvotes: 1

Related Questions