CODEWITHSUNDEEP
iphoneiosios4avcapturesession
Riley Testut
Riley Testut

Reputation: 451

captureOutput:didOutputSampleBuffer:fromConnection Performance Issues

I use AVCaptureSessionPhoto to allow the user to take high-resolution photos. Upon taking a photo, I use the captureOutput:didOutputSampleBuffer:fromConnection: method to retrieve a thumbnail at the time of capture. However, although I try to do minimal work in the delegate method, the app becomes sort of laggy (I say sort of because it is still useable). Also, the iPhone tends to run hot.

Is there some way of reducing the amount of work the iPhone has to do?

I set up the AVCaptureVideoDataOutput by doing the following:

self.videoDataOutput = [[AVCaptureVideoDataOutput alloc] init]; 
self.videoDataOutput.alwaysDiscardsLateVideoFrames = YES;

// Specify the pixel format
dispatch_queue_t queue = dispatch_queue_create("com.myapp.videoDataOutput", NULL);
[self.videoDataOutput setSampleBufferDelegate:self queue:queue];
dispatch_release(queue);
self.videoDataOutput.videoSettings = [NSDictionary dictionaryWithObject: [NSNumber numberWithInt:kCVPixelFormatType_32BGRA] 
                                                   forKey:(id)kCVPixelBufferPixelFormatTypeKey];

Here's my captureOutput:didOutputSampleBuffer:fromConnection (and assisting imageRefFromSampleBuffer method):

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
   fromConnection:(AVCaptureConnection *)connection {

NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
if (videoDataOutputConnection == nil) {
    videoDataOutputConnection = connection;
}
if (getThumbnail > 0) {
    getThumbnail--;
    CGImageRef tempThumbnail = [self imageRefFromSampleBuffer:sampleBuffer];
    UIImage *image;
    if (self.prevLayer.mirrored) {
        image = [[UIImage alloc] initWithCGImage:tempThumbnail scale:1.0 orientation:UIImageOrientationLeftMirrored];
    }
    else {
        image = [[UIImage alloc] initWithCGImage:tempThumbnail scale:1.0 orientation:UIImageOrientationRight];
    }
    [self.cameraThumbnailArray insertObject:image atIndex:0];
    dispatch_async(dispatch_get_main_queue(), ^{
        self.freezeCameraView.image = image;
    });
    CFRelease(tempThumbnail);
}
sampleBuffer = nil;
[pool release];

}

-(CGImageRef)imageRefFromSampleBuffer:(CMSampleBufferRef)sampleBuffer {

CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); 
CVPixelBufferLockBaseAddress(imageBuffer,0); 
uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(imageBuffer); 
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); 
size_t width = CVPixelBufferGetWidth(imageBuffer); 
size_t height = CVPixelBufferGetHeight(imageBuffer); 

CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); 
CGContextRef context = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); 
CGImageRef newImage = CGBitmapContextCreateImage(context); 
CVPixelBufferUnlockBaseAddress(imageBuffer,0);
CGContextRelease(context); 
CGColorSpaceRelease(colorSpace);
return newImage;

}

Upvotes: 2

Views: 5065

Answers (2)

Phien Tram
Phien Tram

Reputation: 381

To improve, we should setup our AVCaptureVideoDataOutput by:

output.minFrameDuration = CMTimeMake(1, 10);

We specify a minimum duration for each frame (play with this settings to avoid having too many frames waiting in the queue because it can cause memory issues). It is similar to the inverse of the maximum frame-rate. In this example we set a min frame duration of 1/10 seconds so a maximum frame-rate of 10fps. We say that we are not able to process more than 10 frames per second.

Hope that help!

Upvotes: 0

andrei
andrei

Reputation: 115

minFrameDuration is deprecated, this may work:

AVCaptureConnection *stillImageConnection = [stillImageOutput connectionWithMediaType:AVMediaTypeVideo];
stillImageConnection.videoMinFrameDuration = CMTimeMake(1, 10);

Upvotes: 1

Related Questions