CODEWITHSUNDEEP
iosffmpegvideo-toolbox
user1784317
user1784317

Reputation: 41

AVSampleBufferDisplayLayer with VTDecompressionSession

I've been struggling with AVSampleBufferDisplayLayer being really choppy with a lot of motion. When there was motion in my live stream, it would be come pixelated and half frozen with multiple frames displaying at once. However, once I added the following piece of code, everything was solved:

VTDecodeFrameFlags flags = kVTDecodeFrame_EnableAsynchronousDecompression 
| kVTDecodeFrame_EnableTemporalProcessing;
VTDecodeInfoFlags flagOut;
VTDecompressionSessionDecodeFrame(decompressionSession, sampleBuffer, flags,
(void*)CFBridgingRetain(NULL), &flagOut);

Note that I did create the decompression session before, but I don't actually do anything in the callback. I am still calling enqueueSampleBuffer: on AVSampleBufferDisplayLayer and that is how the video is displayed on the screen.

Do you have to call VTDecompressionSessionDecodeFrame for AVSampleBufferDisplayLayer to display correctly? I thought AVSampleBufferDisplayLayerr would use VTDecompressionSessionDecodeFrame internally. Is this due to being on the iOS Simulator?

Upvotes: 2

Views: 1425

Answers (2)

fernandowei
fernandowei

Reputation: 55

Of course you can use them together, in which case VTDecompressionSessionDecodeFrame is just for decoding and AVSampleBufferDisplayLayer is just for displaying the VTDecompressionSessionDecodeFrame's output(CVPixelBufferRef, which should be turned into CMSampleBufferRef before enqueuing it into AVSampleBufferDisplayLayer).

Upvotes: 1

Kaleb
Kaleb

Reputation: 1885

AVSampleBufferDisplayLayer and VTDecompressionSession are two different things, although AVSampleBufferDisplayLayer may use VTDecompressionSession under the hood (I don't know). You don't use them together, it is a one or the other where VTDecompressionSession is lower level than AVSampleBufferDisplayLayer. You do not and should not call VTDecompressionSessionDecodeFrame() to work with AVSampleDisplayLayer enqueueSampleBuffer().

From your description, it sounds like you have a timing issue with your PTS (presentation time stamps). Make sure you set up your time base correctly.

See here: Set rate at which AVSampleBufferDisplayLayer renders sample buffers

Also, make sure you are feeding AVSampleBufferDisplayLayer frames in encoder order (the order they came out of the encoder in), not display order.

Finally, make sure you aren't dropping packets somewhere over the network, especially if you are using UDP.

Upvotes: 1

Related Questions