CODEWITHSUNDEEP
opencvopticalflowoutliers
joanna
joanna

Reputation: 773

Remove outliers from Lucas-Kanade optical flow

There are similar questions on SO, but I didn't find the answer I wanted. I need to implement a robust optical flow in order to track features on a (detected) face. I use goodFeaturesToTrack/SURF (I haven't yet decided which is best) to get the initial features.

My question is how can I remove the outliers generated from optical flow? Is RANSAC a valid option for this and if so, how can you combine it with calcOpticalFlowPyrLK?

I also thought of rejecting the features for which the displacement is bigger than a threshold, but it's just an idea and don't really know how to implement it (how to choose the threshold, should I compute the mean displacement, etc). So, which approach is best ?

Upvotes: 4

Views: 3352

Answers (2)

phillies
phillies

Reputation: 11

If you would use a grid of flowpoints instead of feature detection then you could asses flowpoints by comparing the results with the surrounding flowpoints. If the distance with the surrounding vectors is too big, you could eliminate them. But doing this with irregular features is rather too expensive.

If you do continous tracking (of the same features) over several frames, you could also add some temporal smoothness assumption. e.g. a tracking vector from frame N to N+1 is likely to be very similar with the vector from N-1 to N and N+1 to N+2.

Generally, it always makes sense to eliminate suspicous vectors by the features already mentioned above: - vectors which are very long - vectors with high error - tracking points with poor gradient (already excluded, if you use corner detection for the features)

Ransac would only work if you are particularly interested in one rather global feature. e.g. the movement of the head. But I guess that's not what you are interested in (otherwise you could probably also just take the mean of all vectors)

Upvotes: 1

Adi Shavit
Adi Shavit

Reputation: 17285

RANSAC is a good and robust option if you have a model that you expect your motion to conform to.
In general LK is local flow and does not have to conform to any (global) motion model, so in many cases RANSAC is inappropriate.

For general flow you might consider:

  1. Symmetric flow: LK flow from A to B give the same results as an independent LK flow from B to A.
  2. Motion bounds: use domain specific knowledge to, e.g. remove motions that are too big, too sparse, too different than neighbors etc.

Upvotes: 3

Related Questions