Cropping AVCaptureVideoPreviewLayer output to a square

Question

I am having issues with my AVCaptureVideoPreviewLayer method when grabbing a cropped UIImage of the viewable screen. Currently it is working but not outputting the correct crop that I require.

I am trying to output a square but it (by the looks of it) seems to be giving the full height and compressing the image.

The before image shows the LIVE screen and the after image shows the image once the capture button has been pressed. You can see that it has been changed vertically to fit the square but the height hasn't been cropped vertically.

Capture Image Code

[stillImageOutput captureStillImageAsynchronouslyFromConnection:videoConnection completionHandler: ^(CMSampleBufferRef imageSampleBuffer, NSError *error) {

    if (imageSampleBuffer != NULL) {

        NSData *imageData = [AVCaptureStillImageOutput jpegStillImageNSDataRepresentation:imageSampleBuffer];
        [self processImage:[UIImage imageWithData:imageData]];

    }
}];

Cropping code

- (void) processImage:(UIImage *)image { //process captured image, crop, resize and rotate

        haveImage = YES;

    CGRect deviceScreen = _previewLayer.bounds;
    CGFloat width = deviceScreen.size.width;
    CGFloat height = deviceScreen.size.height;

    NSLog(@"WIDTH %f", width); // Outputing 320
    NSLog(@"HEIGHT %f", height); // Outputting 320

    UIGraphicsBeginImageContext(CGSizeMake(width, width));
    [image drawInRect: CGRectMake(0, 0, width, width)];

    UIImage *smallImage = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();

    CGRect cropRect = CGRectMake(0, 0, width, width);

    CGImageRef imageRef = CGImageCreateWithImageInRect([smallImage CGImage], cropRect);

    CGImageRelease(imageRef);

    [captureImageGrab setImage:[UIImage imageWithCGImage:imageRef]];

}

Answer 1

My previewLayer is square and this did nothing for me:

previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer?.videoGravity = AVLayerVideoGravityResizeAspectFill

@Rufel's answer is what WORKED!!! Here it is in Swift 5

func processImageIntoSquare(image: UIImage) -> UIImage? {
    
    guard let previewLayer = previewLayer else { return nil }
    
    let width = previewLayer.bounds.size.width
    
    let size = CGSize(width: width, height: width)
    UIGraphicsBeginImageContext(size)
    
    let imageHeight = floor(width / image.size.width * image.size.height)
    let offSetY = floor((imageHeight - width) / 2.0)
    
    let rect = CGRect(x: 0, y: -offSetY, width: width, height: imageHeight)
    image.draw(in: rect)
    
    let smallImage = UIGraphicsGetImageFromCurrentImageContext()
    UIGraphicsEndImageContext()
    
    return smallImage
}

And here is how I use it:

MyCameraController: AVCapturePhotoCaptureDelegate {
    
    func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {

        if let error = error { return }

        guard let imageData = photo.fileDataRepresentation(), let imageFromCamera = UIImage(data: imageData) else { return }

        guard let squareImage = processImageIntoSquare(image: imageFromCamera) else { return }

        DispatchQueue.main.async { [weak self] in

            self?.myImageView.image = squareImage
        }
    }
}

Answer 2

Have you tried to set videoGravity?

previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer?.videoGravity = AVLayerVideoGravityResizeAspectFill

Answer 3

The aspect ratio is distorted because the image captured by the camera is not square. When drawing it into a square it will be drawn with a distorted aspect ratio. A possible solution is to draw the image into the square preserving the aspect ration like:

- (void) processImage:(UIImage *)image { //process captured image, crop, resize and rotate

    haveImage = YES;

    CGRect deviceScreen = _previewLayer.bounds;
    CGFloat width = deviceScreen.size.width;
    CGFloat height = deviceScreen.size.height;

    NSLog(@"WIDTH %f", width); // Outputing 320
    NSLog(@"HEIGHT %f", height); // Outputting 320

    UIGraphicsBeginImageContext(CGSizeMake(width, width));

    CGFloat aspect_h = image.size.height * width / image.size.width;

    [image drawInRect: CGRectMake(0, -(aspect_h - width)/2.0, width, aspect_h)];

    UIImage *smallImage = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();

    CGRect cropRect = CGRectMake(0, 0, width, width);

    CGImageRef imageRef = CGImageCreateWithImageInRect([smallImage CGImage], cropRect);

    CGImageRelease(imageRef);

}

the new image will be of width width. For that width the height that will preserve the aspect ratio is aspect_h. Then, the image is shifted along the y axis to center it vertically.

Notice that if your app were to work on orientations that are not portrait a little more work is needed.

Another observation: you can get a better resolution image if you use image.size.width as the width for the new image context.

hope this helps!

Answer 4

Even if the preview layer is square, keep in mind that the generated still image keeps its original size.

From what I see, the problem is here:

UIGraphicsBeginImageContext(CGSizeMake(width, width));
[image drawInRect: CGRectMake(0, 0, width, width)];

You already made your context square with the first line. You still need to draw your image in its original format, it will be clipped by that context. On the second line, you are forcing the drawing of the original image in a square, thus making it look "squeezed".

You should find the right image height that keeps the original ratio while fitting your "width". Next, you will want to draw that image with the correct size (keeping the original ratio) in your square context. If you want to clip the center, change your Y position of the drawing.

Something similar to this:

- (void) processImage:(UIImage *)image {
    UIGraphicsBeginImageContext(CGSizeMake(width, width));
    CGFloat imageHeight = floorf(width / image.width * image.height);
    CGFloat offsetY = floorf((imageHeight - width) / 2.0f);
    [image drawInRect: CGRectMake(0, -offsetY, width, imageHeight)];
    UIImage *smallImage = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();
    [captureImageGrab setImage:smallImage];
}

That should do it.

Answer 5

Looking at the code it appears to me that you are drawing the image into a square. This is the part that shrinks the image without regard to the aspect ratio.

Instead, do something like this:

- (void) processImage:(UIImage *)image { //process captured image, crop, resize and rotate

    haveImage = YES;

    CGRect deviceScreen = _previewLayer.bounds;
    CGFloat width = deviceScreen.size.width;
    CGFloat height = deviceScreen.size.height;

    NSLog(@"WIDTH %f", width); // Outputing 320
    NSLog(@"HEIGHT %f", height); // Outputting 320

    CGRect cropRect = CGRectZero;
    if (image.size.width > image.size.height) {
        cropRect.origin.x = (image.size.width-image.size.height)/2;
        cropRect.size = CGSizeMake(image.size.height, image.size.height);
    } else {
        cropRect.origin.y = (image.size.height-image.size.width)/2;
        cropRect.size = CGSizeMake(image.size.width, image.size.width);
    }
    CGImageRef croppedImage = CGImageCreateWithImageInRect(image.CGImage, cropRect);

    UIGraphicsBeginImageContext(CGSizeMake(width, width));
    [[UIImage imageWithCGImage:croppedImage] drawInRect:CGRectMake(0, 0, width, width)];
    UIImage *smallImage = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();

    CGImageRelease(croppedImage);

    [captureImageGrab setImage:[UIImage imageWithCGImage:smallImage.CGImage scale:image.scale orientation:image.imageOrientation]];
}