CODEWITHSUNDEEP
encodingcompressionh.264predictionvideo-codecs
Leandro Moreira
Leandro Moreira

Reputation: 377

How does H264 inter prediction fills the prediction block old position?

In my understanding (a very simplistic view), the inter prediction (motion estimation / compensation) of H264 standard first finds the best match block on the reference frame, and then it's encoded with motion vector (the effective new X and Y) and the residual (prediction vs reality).

But how does the decoder knows how to fill the old space where the predicted block was before? I'm supposing that the residual is calculated from its new position, on a block level, not a frame level.

Let's say the encoder decided to use inter prediction to the encode the following two images, it calculates where does the ball should be (its new position and the residual energy) but how does it fill the old space?

frame 0 frame 1

Upvotes: 2

Views: 881

Answers (1)

osgx
osgx

Reputation: 94345

Motion compensation is just optimization of frame encoding. If we talk about motion vectors, this is "block motion compensation", as defined in Wikipedia:

Block motion compensation divides up the current frame into non-overlapping blocks, and the motion compensation vector tells where those blocks come from (a common misconception is that the previous frame is divided up into non-overlapping blocks, and the motion compensation vectors tell where those blocks move to). The source blocks typically overlap in the source frame. Some video compression algorithms assemble the current frame out of pieces of several different previously-transmitted frames.

So, motion vector is not "move the block from old frame", it is (intra-) encoding of current frame's macroblocks, when some blocks are copied from older frame with some small shift (and some blocks can be copied several times from previous frame into current; and most block are copied with zero motion vector). In theory we can encode new frame by its macroblocks, but with help of motion compensation we have change to get lot of image information from previous frames and encode less. Parts of images which are not compensated are encoded with image macroblocks.

Example from other Wikipedia page from "Big Buck Bunny" free film:

Frame with motion vectors displayed, dots are zero vectors

Residual image after motion compensation to be encoded

There is good description of the encoding/decoding processes in H.264 but in russian: http://www.compression.ru/dv/course/compr_h264.pdf (from http://www.compression.ru/video/ site)

And description of motion compensation in English: http://inst.eecs.berkeley.edu/~ee290t/sp04/lectures/02-Motion_Compensation_girod.pdf (prediction error is encoded, and completely new image will be is almost fully mispredicted. Background image has low frequency of information and most probably will be motion compensated by some of nearby background block.)

Upvotes: 3

Related Questions