CODEWITHSUNDEEP
iosobjective-cxcode10decoderiosurface
Ravi
Ravi

Reputation: 46

VTDecompressionSession - IOSurface allocations gradually increase

I am using VTDecompressionSession to decode a .h264 stream. The decoder works as expected and I get properly decoded buffers back. However, I see a gradual increase in "Created And Persistent" allocations in XCode instrument "Allocations". As shown in the screenshot, these can be attributed to IOSurface buffers that the decoder allocates internally and these wont get released even after the VTDecompressionSession is released. I saw these happening in both synchronous decoding and async callbacks decompressionSessionDecodeFrameCallback. The number of frames left over are random in occurrence and time. The size of these buffers is exactly equal to the size of the decoded frame. I do call VTDecompressionSessionWaitForAsynchronousFrames before invalidating the decoder session, but these allocations dont go away. Is there a way to release these IOSurface buffers when the decoder session ends ?

This is the outline of my decoder run. Create a decoder session

const void *values[] = { CFNumberCreate(NULL, kCFNumberSInt32Type, &v) };       
attrs                = CFDictionaryCreate(NULL, keys, values, 1, NULL, NULL);


VTDecompressionOutputCallbackRecord callBackRecord;
callBackRecord.decompressionOutputCallback = decompressionSessionDecodeFrameCallback;
callBackRecord.decompressionOutputRefCon = (__bridge void *)self;
VTDecompressionSessionCreate(kCFAllocatorDefault,
                                          _decoderFormatDescription,
                                          NULL,   
                                          attrs,
                                          &callBackRecord,
                                          &_decoderSession);

Call to do the decoding when the a NALU packet is ready

CMSampleBufferRef sampleBuffer = nil;
const size_t sampleSizeArray[] = {packetLen};    
CMSampleTimingInfo sampleTimeinfo ={CMTimeMake(1,FPS), CMTimeMake(presentationTS, 1000000), kCMTimeInvalid};
CMSampleBufferCreateReady(kCFAllocatorDefault, blockBuffer, _decoderFormatDescription ,
                                       1, 1, &sampleTimeinfo, 1, sampleSizeArray, &sampleBuffer);
flags = kVTDecodeFrame_EnableAsynchronousDecompression;

VTDecompressionSessionDecodeFrame(_decoderSession,
                                          sampleBuffer,
                                          flags,
                                          &sampleBuffer,
                                          &flagOut);

Callback 

void decompressionSessionDecodeFrameCallback{

    CVPixelBufferLockBaseAddress(imageBuffer,0);
    .... 
    send to display
    ....
    CVPixelBufferUnlockBaseAddress(imageBuffer,0);
}

Ending decoder session

VTDecompressionSessionWaitForAsynchronousFrames(_decoderSession);
VTDecompressionSessionInvalidate(_decoderSession);
CFRelease(_decoderSession);

enter image description here

Upvotes: 2

Views: 945

Answers (1)

Declan Stewart McPartlin
Declan Stewart McPartlin

Reputation: 171

That is not normal, there is certainly a leak in your code.

This is not an issue with releasing the decompressionSession, such a memory leak is different, that becomes apparent when closing and opening streams.

I noticed you referenced the sampleBuffer as the sourceFrameRefCon in the call:

VTDecompressionSessionDecodeFrame(_decoderSession,
                                          sampleBuffer,
                                          flags,
                                          &sampleBuffer, //here
                                          &flagOut);

This is wrong, unless you'd like to provide a pointer there to be able to reference the frame itself (not the buffer), leave this value as NULL. This could very well be your issue.

Since we can only see a portion of your code, I will point out where other issues like this one usually occur.

If you ever call:

CVBufferRetain(imageBuffer);

or

CFRetain(imageBuffer);

Then you must also call:

CVBufferRelease(imageBuffer);

or

CFRelease(imageBuffer);

after, respectively.

Be certain that the player receiving the buffer does not retain the buffer in any way, on receiving the buffer it should immediately copy the buffer to its own buffer, whatever that may be (unless you are using a metal buffer to directly provide the buffer to the display mechanism, in which case you should release the buffer after being displayed).

Upvotes: 0

Related Questions