CODEWITHSUNDEEP
androidimage-processingvideovideo-streamingandroid-mediacodec
Mary H
Mary H

Reputation: 409

Android MediaCodec's getOutputImage() returns a read-only image

I have a video player that uses MediaCodec to render frames to a surface, just as suggested here, and it works. I need to apply some processing to the frames and change their pixels before they are rendered. But getOutputImage() returns a Read-Only image. How can I change the pixel data then?

android. media. Image getOutputImage( int index ) Returns a read-only Image object for a dequeued output buffer index that contains the raw video frame. After calling this method, any ByteBuffer or Image object previously returned for the same output index MUST no longer be used.

Here is my snippet:

Image image = codec.getOutputImage(outputBufferIndex);
if (image != null) {
    applyCustomProcessing(image);
     image.close();
}
SurfaceRenderer surfaceRenderer = (SurfaceRenderer) renderer;
surfaceRenderer.render(mediacodec, outputBufferIndex, presentationTime);

Upvotes: 0

Views: 73

Answers (1)

dev.bmax
dev.bmax

Reputation: 10621

Video frames usually have a format that is not very convenient for image processing (YCbCr color space, planar or semi-planar memory layout, chroma subsampling).

Typically you would first convert it to an RGB image.

You can do it manually. Search for: "convert YUV to RGB". Note that you need to handle multiple possible input formats. This is also relatively slow.

An alternative way is to use a SurfaceTexture and OpenGL ES to do the processing on the GPU. In this case the color conversion happens automatically and you get a performance boost. But this requires writing some amount of setup code and implementing your image processing algorithms in OpenGL.

Upvotes: 1

Related Questions