CODEWITHSUNDEEP
algorithmmathvideocomputer-vision
skrat
skrat

Reputation: 5562

Obtaining motion vectors from raw video

I'd like to know if there is any good (and freely available) text, on how to obtain motion vectors of macro blocks in raw video stream. This is often used in video compression, although my application of it is not video encoding.

Code that does this is available in OSS codecs, but understanding the method by reading the code is kinda hard.

My actual goal is to determine camera motion in 2D projection space, assuming the camera is only changing it's orientation (NOT the position). What I'd like to do is divide the frames into macro blocks, obtain their motion vectors, and get the camera motion by averaging those vectors.

I guess OpenCV could help with this problem, but it's not available on my target platform.

Upvotes: 2

Views: 2198

Answers (4)

jilles de wit
jilles de wit

Reputation: 7138

For the most low-level algorithms of this type the term you are looking for is optical flow and one of the easiest algorithms of that class is the Lucas Kanade algorithm.

This is a pretty good overview presentation that should give you plenty of ideas for an algorithm that does what you need

Upvotes: 0

Martin Thompson
Martin Thompson

Reputation: 16792

Sounds like you're doing a very limited SLAM project?

Lots of reading matter at Bristol University, Imperial College, Oxford University for example - you might find their approaches to finding and matching candidate features from frame to frame of interest - much more robust than simple sums of absolute differences.

Upvotes: 0

phkahler
phkahler

Reputation: 5767

If you assume only camera motion, I suspect there is something possible with analysis of the FFT of successive images. For frequencies whose amplitudes have not changed much, the phase information will indicate the camera motion. Not sure if this will help with camera rotation, but lateral and vertical motion can probably be computed. There will be difficulties due to new information appearing on one edge and disappearing on the other and I'm not sure how much that will hurt. This is speculative thinking in response to your question, so I have no proof or references :-)

Upvotes: 0

Jerry Coffin
Jerry Coffin

Reputation: 490108

The usual way is simple brute force: Compare a macro block to each macro block from the reference frame and use the one that gives the smallest residual error. The code gets complex primarily because this is usually the slowest part of mv-based compression, so they put a lot of work into optimizing it, often at the expense of anything even approaching readability.

Especially for real-time compression, some reduce the workload a bit by (for example) restricting the search to the original position +/- some maximum delta. This can often gain quite a bit of compression speed in exchange for a fairly small loss of compression.

Upvotes: 3

Related Questions