MPEG backwards frame decoding using FFmpeg

Question

I have so-called "block's" that stores some of MPEG4 (I,p,p,p,p...) frames. For every "block", frame starts with an "I" frame, and ends before next "I" frame. (VOL - "visual_object_sequence_start_code" is allways included before the "I" frame)

I need to be able to play those "block" frames in "backwards" mode. The thick is that:

1. It's not possible to just take the last frame in my block and perform decoding, because it's a "P" frame and it needs an "inter frame (I)" to be correctly decoded.

2. I can't just take my first "I" frame, then pass it to the ffmpeg's "avcodec_decode_video" function and only then pass my last "P" frame to the ffmpeg, because that last "P" frame depends on the "P" frame before it, right? (well.. as far as I've tested this method, my last decoded P frame had artifacts)

Now the way I'm performing backwards playing is - first decoding all of my "block" frames in RGB and store them in memory. (in most cases it would be ~25 frames per block max.) But this method really requires a lot of memory... (especially if frames resolutions is high) And I have a feeling that this is not the right way to do this...

So I would like to ask, does any one have any suggestions how this "backwards" frame decoding/playing could be performed using FFmpeg?

Thanks

Answer 1

What you are looking at really a research problem: To get a glimps of the overall approach, look at the following paper:

1. Compressed-Domain Reverse Play of MPEG Video Streams, SPIE International Symposium on Voice, Video, and Data Communications, Boston, MA, November, 1998.

2. Reverse-play algorithm for MPEG video streaming

3. MANIPULATING TEMPORAL DEPENDENCIES IN COMPRESSED VIDEO DATA WITH APPLICATIONS TO COMPRESSED-DOMAIN PROCESSING OF MPEG VIDEO.

Essentially, there is still advanced encoding based on key frames, however, you can reverse the process of motion compensation to achieve the reverse flow. This is done by on the fly conversion of P frames into I frames. This does require looking forward but doesn't require that much more memory. Possibly you can save this as a new file and then apply it to standard decoder with reverse play requirements.

However, this is very complex, and i have seen rare softwares doing this practically.

Answer 2

I do not think there is a way around starting from I-frame and decoding all P-frames, due to P-frame depending on the previous frame. To handle decoded frames, they can be saved to a file, or, with limited storage and extra CPU power, older P-frames can be discarded and recomputed later.

At the command level, you can convert input video to a series of images:

ffmpeg -i input_video output%4d.jpg

then reverse their order somehow and convert back to a video:

ffmpeg -r FRAME_RATE -i reverse_output%4d.jpg output_video

You may consider pre-processing, if it is an option.