Jordi Mares Soler
Jordi Mares Soler

Reputation: 31

Orientation problems in captureOutput:didOutputSampleBuffer:from connection:

I am trying to process each frame of an AVCaptureSession and previewing the filtered image in an UIImageView. It works but the images in the UIImageView appear rotated (and distorted). I have been trying to find answers here and in Google but I could not find any working solution... Does anyone have any idea on what to try?

Btw, I am using Swift 3.0

This is the code I am using:

func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!) {

    let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
    let attachments = CMCopyDictionaryOfAttachments(kCFAllocatorMalloc, sampleBuffer!, kCMAttachmentMode_ShouldPropagate)
    let coreImage = CIImage(cvImageBuffer: pixelBuffer!, options: attachments as! [String : Any]?)
    var filteredImage = UIImage(ciImage: coreImage, scale: 1.0, orientation: UIImageOrientation.right)

    if filter {
        NSLog("FILTER ACTIVATED")
        let filterType = CIFilter(name: "CISepiaTone")
        filterType?.setValue(coreImage, forKey: kCIInputImageKey)
        filterType?.setValue(0.5, forKey: kCIInputIntensityKey)
        filteredImage = UIImage(ciImage: filterType!.value(forKey: kCIOutputImageKey) as! CIImage!, scale: filteredImage.scale, orientation: UIImageOrientation.right)
    }

    DispatchQueue.main.async() {
        self.imageViewPreview.image = filteredImage // UIImageView
    }
}

And this is what I see in the preview:

UIImageView preview

Many thanks in advance

Upvotes: 1

Views: 1307

Answers (2)

M-P
M-P

Reputation: 4959

For iOS 17+:

Create a rotationCoordinator variable:

private var rotationCoordinator: AVCaptureDevice.RotationCoordinator?

Assign the rotationCoordinator in configurePreviewLayer():

func configurePreviewLayer(_ view: UIView) {
    guard let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera,
                                                    for: .video,
                                                    position: .back) else { return }
        
    previewLayer = AVCaptureVideoPreviewLayer(session: session)
    previewLayer?.videoGravity = .resizeAspectFill
        
    guard let previewLayer = previewLayer else { return }
    view.layer.addSublayer(previewLayer)
        
    rotationCoordinator = AVCaptureDevice.RotationCoordinator(device: videoDevice, previewLayer: previewLayer)
}

Then use it to assign a videoRotationAngle to the connection in captureOutput():

func captureOutput(_ output: AVCaptureOutput,
                      didOutput sampleBuffer: CMSampleBuffer,
                      from connection: AVCaptureConnection) {
    guard let coordinator = rotationCoordinator else { return }
    connection.videoRotationAngle = coordinator.videoRotationAngleForHorizonLevelCapture
        
    guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
        
    let ciImage = CIImage(cvImageBuffer: imageBuffer)
    let context = CIContext()
    guard let cgImage = context.createCGImage(ciImage, from: ciImage.extent) else { return }
        
    currentFrame = UIImage(cgImage: cgImage)
}

Note the use of videoRotationAngleForHorizonLevelCapture and NOT videoRotationAngleForHorizonLevelPreview.

Upvotes: 0

Jordi Mares Soler
Jordi Mares Soler

Reputation: 31

I found it! It turned out I had to add the line connection.videoOrientation = AVCaptureVideoOrientation.portrait in that method...

Upvotes: 1

Related Questions