Correctly set the right picture orientation when shooting photo

Question

On my iOS app written in Swift I need to take pictures and save them on gallery; as Apple documentation, all the pictures are taken in landscape also if the phone is in portrait; if we save the picture as-is it will be saved 90° rotated.

The question: How can I correctly manage the device orientation when saving picture?

Thank to some searches I used this solution:

    func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
        // ...error handling here

        guard let imageData = photo.fileDataRepresentation(with: self) else {
            NSLog("Fail to convert pixel buffer")
            return
        }
        
        // Save image to gallery here
    }

My class is AVCapturePhotoFileDataRepresentationCustomizer delegate, so:

    func replacementMetadata(for photo: AVCapturePhoto) -> [String : Any]? {
        var properties = photo.metadata

        // Image orientation
        properties[kCGImagePropertyOrientation as String] = CGImagePropertyOrientation.right
        
        let exifData = NSMutableDictionary(dictionary: properties[kCGImagePropertyExifDictionary as String] as! NSDictionary)
        let xDimension = exifData[kCGImagePropertyExifPixelYDimension as String]
        let yDimension = exifData[kCGImagePropertyExifPixelXDimension as String]
        
        if xDimension != nil && yDimension != nil {
            exifData[kCGImagePropertyExifPixelXDimension as String] = xDimension
            exifData[kCGImagePropertyExifPixelYDimension as String] = yDimension
            
            properties[kCGImagePropertyExifDictionary as String] = exifData
        }
        
        return properties
    }

Because the image is shot in portrait the orientation is .right and I read in my searches that also the X and Y dimensions in exif data should be swapped.

Unfortunately everything lead no result: Inspecting the saved image with an exif explorer image orientation is still a 0=unknown value and X and Y as originally set.

I'm sure the data is correctly set and written because:

1. Breakpoing on return properties underlined that the orientation tag set correctly; moreover if a tag is not correctly set or unknown the app crashes...

2. In the same replacementMetadata function I also set GPS data (I cut this for simplicity here!) and I wrote some test value (like heading = 101) and these data are correctly reported on the final image metadata.

So my question remains ... Thank you for pointing me in the right direction with code snippets or documents.

Answer 1

You could use these functions

private var deviceOrientation: UIDeviceOrientation {
    var orientation = UIDevice.current.orientation
    if orientation == UIDeviceOrientation.unknown {
        orientation = UIScreen.main.orientation
    }
    return orientation
}

private func videoOrientationFor(_ deviceOrientation: UIDeviceOrientation) -> AVCaptureVideoOrientation? {
    switch deviceOrientation {
    case .portrait: return AVCaptureVideoOrientation.portrait
    case .portraitUpsideDown: return AVCaptureVideoOrientation.portraitUpsideDown
    case .landscapeLeft: return AVCaptureVideoOrientation.landscapeRight
    case .landscapeRight: return AVCaptureVideoOrientation.landscapeLeft
    default: return nil
    }
}

fileprivate extension UIScreen {

var orientation: UIDeviceOrientation {
    let point = coordinateSpace.convert(CGPoint.zero, to: fixedCoordinateSpace)
    if point == CGPoint.zero {
        return .portrait
    } else if point.x != 0 && point.y != 0 {
        return .portraitUpsideDown
    } else if point.x == 0 && point.y != 0 {
        return .landscapeRight //.landscapeLeft
    } else if point.x != 0 && point.y == 0 {
        return .landscapeLeft //.landscapeRight
    } else {
        return .unknown
    }
}

}

and use it in your function takePhoto

        if let photoOutputVideoConnection = photoOutput.connection(with: .video) {
            if photoOutputVideoConnection.isVideoOrientationSupported,
                let videoOrientation = self.videoOrientationFor(self.deviceOrientation) {
                photoOutputVideoConnection.videoOrientation = videoOrientation
            }
        }
        
        photoOutput.capturePhoto(with: photoSettings, delegate: self)

You can't change the value of metadata, the value is read only, you have in the metadata a value to know the orientation, and the property indicates which transform was applied to display correctly your photo

Answer 2

let deviceOrientation : UIDeviceOrientation =
UIDevice.current.orientation

var avcaptureOrientation : AVCaptureVideoOrientation = .landscapeRight

if (deviceOrientation == .landscapeLeft) {
    
    avcaptureOrientation = .landscapeRight
    
} else if (deviceOrientation == .landscapeRight ) {
    
    avcaptureOrientation = .landscapeLeft
    
} else if (deviceOrientation == .portrait) {
    
    avcaptureOrientation = .portrait
    
} else if (deviceOrientation == .portraitUpsideDown) {
    
    avcaptureOrientation = .portraitUpsideDown
    
}

self.previewLayer?.connection?.videoOrientation = avcaptureOrientation

self.photoOutput.connection(with: .video)?.videoOrientation = avcaptureOrientation

Answer 3

import Foundation
import CoreMotion
import AVFoundation
import UIKit

protocol OrientationHandlerDelegate: class {
   func didChange(deviceOrientation: UIDeviceOrientation)
}

class OrientationHandler {

  private let motionManager = CMMotionManager()
  private let queue = OperationQueue()
  private var deviceOrientation: UIDeviceOrientation = .unknown
  weak var delegate: OrientationHandlerDelegate?

  init() {
    motionManager.accelerometerUpdateInterval = 1.0
    motionManager.deviceMotionUpdateInterval = 1.0
    motionManager.gyroUpdateInterval = 1.0
    motionManager.magnetometerUpdateInterval = 1.0
  }

  func startMeasuring() {
    guard motionManager.isDeviceMotionAvailable else {
        return
    }
    motionManager.startAccelerometerUpdates(to: queue) { [weak self] (accelerometerData, error) in
        guard let strongSelf = self else {
            return
        }
        guard let accelerometerData = accelerometerData else {
            return
        }

        let acceleration = accelerometerData.acceleration
        let xx = -acceleration.x
        let yy = acceleration.y
        let z = acceleration.z
        let angle = atan2(yy, xx)
        var deviceOrientation = strongSelf.deviceOrientation
        let absoluteZ = fabs(z)

        if deviceOrientation == .faceUp || deviceOrientation == .faceDown {
            if absoluteZ < 0.845 {
                if angle < -2.6 {
                    deviceOrientation = .landscapeRight
                } else if angle > -2.05 && angle < -1.1 {
                    deviceOrientation = .portrait
                } else if angle > -0.48 && angle < 0.48 {
                    deviceOrientation = .landscapeLeft
                } else if angle > 1.08 && angle < 2.08 {
                    deviceOrientation = .portraitUpsideDown
                }
            } else if z < 0 {
                deviceOrientation = .faceUp
            } else if z > 0 {
                deviceOrientation = .faceDown
            }
        } else {
            if z > 0.875 {
                deviceOrientation = .faceDown
            } else if z < -0.875 {
                deviceOrientation = .faceUp
            } else {
                switch deviceOrientation {
                case .landscapeLeft:
                    if angle < -1.07 {
                        deviceOrientation = .portrait
                    }
                    if angle > 1.08 {
                        deviceOrientation = .portraitUpsideDown
                    }
                case .landscapeRight:
                    if angle < 0 && angle > -2.05 {
                        deviceOrientation = .portrait
                    }
                    if angle > 0 && angle < 2.05 {
                        deviceOrientation = .portraitUpsideDown
                    }
                case .portraitUpsideDown:
                    if angle > 2.66 {
                        deviceOrientation = .landscapeRight
                    }
                    if angle < 0.48 {
                        deviceOrientation = .landscapeLeft
                    }
                case .portrait:
                    if angle > -0.47 {
                        deviceOrientation = .landscapeLeft
                    }
                    if angle < -2.64 {
                        deviceOrientation = .landscapeRight
                    }
                default:
                    if angle > -0.47 {
                        deviceOrientation = .landscapeLeft
                    }
                    if angle < -2.64 {
                        deviceOrientation = .landscapeRight
                    }
                }
            }
        }
        if strongSelf.deviceOrientation != deviceOrientation {
            strongSelf.deviceOrientation = deviceOrientation
            strongSelf.delegate?.didChange(deviceOrientation: deviceOrientation)
        }
    }
  }

  func stopMeasuring() {
    motionManager.stopAccelerometerUpdates()
  }

  func currentInterfaceOrientation() -> AVCaptureVideoOrientation {
    switch deviceOrientation {
    case .portrait:
        return .portrait
    case .landscapeRight:
        return .landscapeLeft
    case .landscapeLeft:
        return .landscapeRight
    case .portraitUpsideDown:
        return .portraitUpsideDown
    default:
        return .portrait
    }
  }
}

In your file having camera code:

let orientationHandler = OrientationHandler()

Start measuring orientation when camera is active.

orientationHandler.delegate = self
orientationHandler.startMeasuring()

Stop measuring orientation when camera is not in use.

orientationHandler.stopMeasuring()
orientationHandler.delegate = nil

Finally add one line just before calling capturePhoto(with: settings, delegate: self)

//Need to correct image orientation before moving further
    if let photoOutputConnection = photoOutput?.connection(with: .video) {
        photoOutputConnection.videoOrientation = orientationHandler.currentInterfaceOrientation()
    }
    photoOutput?.capturePhoto(with: settings, delegate: self)

Answer 4

You may set your shot orientation to whatever you like by setting videoOrientation of your AVCapturePhotoOutput.

To match it with the current device orientation, you may use UIDevice.current.orientation manually converted to AVCaptureVideoOrientation.

let photoOutput = AVCapturePhotoOutput()

func takeShot() {

    // set whatever orientation you like
    let myShotOrientation = UIDevice.current.orientation.asCaptureVideoOrientation

    if let photoOutputConnection = self.photoOutput.connection(with: .video) {
        photoOutputConnection.videoOrientation = myShotOrientation
    }

    photoOutput.capturePhoto(...)
}

Conversion from UIDeviceOrientation to AVCaptureVideoOrientation:

extension UIDeviceOrientation {
    
    ///
    var asCaptureVideoOrientation: AVCaptureVideoOrientation {
        switch self {
        // YES, that's not a mistake
        case .landscapeLeft: return .landscapeRight
        case .landscapeRight: return .landscapeLeft
        case .portraitUpsideDown: return .portraitUpsideDown
        default: return .portrait
        }
    }
}