CODEWITHSUNDEEP
androidvideoandroid-mediacodec
Harkish
Harkish

Reputation: 2372

Android app that uses MediaSync freezes while playing video

I am trying to use the new MediaSync API to play video and Audio in sync . https://developer.android.com/reference/android/media/MediaSync.html

I have the Audio Only player working using MediaSync,but for the Video only player, I get this in the logcat right after 5-6 frames are displayed

I/MediaSync﹕ still waiting to release a buffer before acquire

My Video only player is as below

public class VideoDecoderTask implements Runnable {
......
@Override
public void run() {

    mediaSync = new MediaSync();
    mediaSync.setSurface(surface);
    Surface inputSurface = mediaSync.createInputSurface();

    mediaExtractor = new MediaExtractor();

    try {
        mediaExtractor.setDataSource(this.clipPath);
    } catch (IOException e1) {
        e1.printStackTrace();
    }

    for (int i = 0; i < mediaExtractor.getTrackCount(); i++) {
        MediaFormat format = mediaExtractor.getTrackFormat(i);
        String mime = format.getString(MediaFormat.KEY_MIME);
        if (mime.startsWith("video/")) {
            Log.d(LOG_TAG, format.toString());
            mediaExtractor.selectTrack(i);
            try {
                videoDecoder = MediaCodec.createDecoderByType(mime);
            } catch (IOException e) {
                e.printStackTrace();
            }
            videoDecoder.configure(format, inputSurface, null, 0);
            Log.d(LOG_TAG, "Found a video track.");
            break;
        }
    }

    SyncParams syncParams = new SyncParams();
    syncParams.setSyncSource(SyncParams.SYNC_SOURCE_VSYNC);
    mediaSync.setPlaybackParams(new PlaybackParams().setSpeed(1.f));
    mediaSync.setSyncParams(syncParams);

    videoDecoder.setCallback(decoderCallback, null);
    videoDecoder.start();
}

MediaCodec.Callback decoderCallback = new MediaCodec.Callback() {
    @Override
    public void onInputBufferAvailable(MediaCodec codec, int index) {
        if (index >= 0) {
            ByteBuffer byteBuffer = codec.getInputBuffer(index);
            int sampleSize = mediaExtractor.readSampleData(byteBuffer, 0);
            Log.d(LOG_TAG, "SampleSize: " + sampleSize);
            if (sampleSize < 0) {
                //we're at end of file so submit EOS to the decoder input
                Log.d(LOG_TAG, "Video Decoder input EOS reached");
                codec.queueInputBuffer(index, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
            } else {
                long sampleTime = mediaExtractor.getSampleTime();
                codec.queueInputBuffer(index, 0, sampleSize, sampleTime, 0);
                mediaExtractor.advance();
            }
        }
    }

    @Override
    public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo info) {
        Log.d(LOG_TAG, "Rendering with preso time: " + info.presentationTimeUs);
        codec.releaseOutputBuffer(index, info.presentationTimeUs);
    }
};

}

The above tasks are kicked off from the main thread using

VideoDecoderTask decoderTask = new VideoDecoderTask(clipPath, new    Surface(surface));
        Thread thread = new Thread(decoderTask);
        thread.start();

where surface is from a TextureView

My educated guess from the message mediasync generates is that for some reason the buffers returned in onOutputBufferAvailable is not released back. What do I need to resolve this?

BTW I am running this on a Nexus9 running dev release M from https://developer.android.com/preview/download.html#images

Upvotes: 0

Views: 1781

Answers (2)

skysign
skysign

Reputation: 1124

I found one problem.

codec.releaseOutputBuffer(index, info.presentationTimeUs);

codec.releaseOutputBuffer(index, 1000 * info.presentationTimeUs);

Please refer the MediaSync example below.

https://github.com/skysign/MediaSyncExample

Upvotes: 1

Harkish
Harkish

Reputation: 2372

If I replace the output TextureView with a SurfaceView, Video playback seems to work fine.

Though this solves my problem, I do not understand why there is a difference between a surface created with a SurfaceTexture and a surface that is obtained from a SurfaceHolder.Per http://developer.android.com/reference/android/graphics/SurfaceTexture.html "A Surface created from a SurfaceTexture can be used as an output destination for the android.hardware.camera2, MediaCodec, MediaPlayer". This is pretty much what I am doing in my above example that uses MediaSync/MEdiaCodec and TextureView as the output surface

Upvotes: 0

Related Questions