ARKit ARCamera transform incorrectly rotated 90 degrees clockwise

Question

I'm following along with one of the older WWDC videos on ARKit found here: https://developer.apple.com/videos/play/wwdc2017/602/

In this video, they create an AR app that listens for a tap gesture and creates an image plane using a snapshot of the scene view.

Here's the relevant code from the video:

However, when I run the code on my device, the generated plane is always rotated 90 degrees clockwise with respect to my device's orientation. For example, when holding my device in portrait mode, the generated plane looks like this:

Similarly, if I hold my device in the landscape-right orientation, the "top" of the image plane will be pointing down.

Is there a reason the ARCamera transform always seems to be off by 90 degrees with respect to the device orientation?

Answer 1

By default, image captured by iPhone or iPad's camera is encoded in the camera sensor's native landscape orientation. When you make a snapshot (even if your device is in portrait orientation), iOS writes an orientation value of:

CGImagePropertyOrientation.right

in the resulting image file. ARKit app reads this value from the file's metadata and then displays the image applying a 90° ClockWise rotation, so your captured image appears in the intended photographer's orientation.

You need to apply the opposite operation – rotate the image 90° CounterClockWise.

So, read this post to find out how to make it using CVPixelBuffer.

Also, the following approach with extensions could help you too:

extension CGImagePropertyOrientation {

    init(_ uiOrientation: UIImage.Orientation) {

        switch uiOrientation {
            case .up: self = .up
            case .upMirrored: self = .upMirrored
            case .down: self = .down
            case .downMirrored: self = .downMirrored
            case .left: self = .left
            case .leftMirrored: self = .leftMirrored
            case .right: self = .right
            case .rightMirrored: self = .rightMirrored
            @unknown default: fatalError()
        }
    }
} 
extension UIImage.Orientation {

    init(_ cgOrientation: UIImage.Orientation) {

        switch cgOrientation {
            case .up: self = .up
            case .upMirrored: self = .upMirrored
            case .down: self = .down
            case .downMirrored: self = .downMirrored
            case .left: self = .left
            case .leftMirrored: self = .leftMirrored
            case .right: self = .right
            case .rightMirrored: self = .rightMirrored
            @unknown default: fatalError()
        }
    }
}