CODEWITHSUNDEEP
computer-visiontrackingopticalflow
Niranjan Kotha
Niranjan Kotha

Reputation: 309

Do we need to computer optical flow in a Kanade-Lucas-Tomasi tracker?

I am trying to understand Kanade-Lucas-Tomasi tracker. This is the overview (I read from some lectures) of how it should be done:

1. Find harris corners 
2. For each corner compute displacement to next frame
using the Lucas-Kanade method
3. Store displacement of each corner, update corner position
4. (optional) Add more corner points every M frames using 1
5. Repeat 2 to 3 (4)
6. Returns long trajectories for each corner point

My doubt in this is do we need to compute the optical flow at some point or just displacement vector is enough to carry out the algorithm? If No then why is optical flow dealt with this topic?

Upvotes: 1

Views: 611

Answers (3)

Stefan Karlsson
Stefan Karlsson

Reputation: 1085

Not the most accurate or helpful description of KLT. This is better:

  1. Find features to track (e.g corners)
  2. For each feature compute displacement to next frame using the Lucas-Kanade method
  3. Store trajectories of the features tracked
  4. (optional) Add and remove features over time
  5. Returns trajectories for each feature

While @Tobias senst answer is not technically wrong, it misses the very important fact:

Lucas-Kanade method is not restricted to estimating displacements. In fact, using LK in the KLT tracker almost assumes that you go for at least rotation and scaling deformation on top of displacement. For many applications, a full affine solution is beneficial. This means you solve for 6 parameters, not only the 2 parameters of displacement (u,v).

Upvotes: 0

Tobias Senst
Tobias Senst

Reputation: 2830

Kanade-Lucas-Tomasi tracker is related to optical flow because the displacement vectors are optical flow vectors but in a sparse sense and not a dense optical flow field. Thats because the tracker is based on the Lucas Kanade method estimating the displacement vector. And Lucas Kanade method is based on the intensity constancy assumption which is solved by a first-order Taylor approximimation this approximation is called optical flow equation and was invented by Horn and Schunk. The Lucas Kanade method is classified as a local optical flow method while most of nowadays most optical flow methods are global methods that produce dense motion fields.

Upvotes: 2

Francesco Callari
Francesco Callari

Reputation: 11825

The displacement vector is one sample of the (dense) optical flow at the corner position. BTW, although Harris corners are usually good location for initializing the search, strictly speaking you need a weaker detector for that, see Shi & Tomasi's classic "good features to track" paper.

Upvotes: 0

Related Questions