CODEWITHSUNDEEP
androidvideo-processingandroid-mediacodec
yarin
yarin

Reputation: 535

Android MediaCodec -long processing for each frame

Edit as i wasn't clear at first time:

I'm trying to use android MediaCodec to get each frame from existing video file(videoBefore.MP4) ,process the frame(like blur) and then encode each frame to a new video file(videoAfter.MP4).
The new video have to be in the same duration as the first.

Just 1 condition:
Every frame should be process with unlimited time,it mean that 10 sec video could take 1 minute for processing.

So far i saw only examples with quick processing (like blue shift) that could be done in real time.
Is there any way to grab the frame from the video,and then "take my time" to process it,and still preserved the new video with the same frame rate or frame timing?

*it could be better if i can preserve the audio too-but the frame is what important.

Thanks!

Upvotes: 0

Views: 1273

Answers (1)

fadden
fadden

Reputation: 52303

You can take as long as you like. The timing of the frames is determined by the presentation time stamp embedded in the .mp4 file, not the rate at which the code is accessed.

You get the time value for each frame from MediaExtractor#getSampleTime(), pass it into the decoder's queueInputBuffer(), and receive it in the BufferInfo struct associated with the decoder's output buffer. Do your processing and submit the frame to the encoder, again specifying the time stamp in queueInputBuffer(). It will be passed through BufferInfo to the output side of the encoder, and you just pass the whole BufferInfo to MediaMuxer#writeSampleData().

You can see the extraction side in ExtractMpegFramesTest and the muxing side in EncodeAndMuxTest. The DecodeEditEncodeTest does the encode/decode preserving the time stamp, but doesn't show the MediaExtractor or MediaMuxer usage.

Bear in mind that the codecs don't really care about time stamps. It's just the extractor/muxer code that handles the .mp4 file that cares. The value gets passed through the codec partly as a convenience, and partly because it's possible for encoded frames to appear out of order. (The decoded frames, i.e. what comes out of the decoder, will always be in order.)

If you fail to preserve the presentation times, you will get video that either lasts zero seconds (and isn't very interesting), or possibly video that lasts a very, very long time. The screenrecord command introduced in Android 4.4 uses the time stamps to avoid recording frames when the screen isn't being updated.

Upvotes: 2

Related Questions