ARKit 3D Head tracking in scene

Question

I am using ARKit to create an augmented camera app. When the ARSession initialises, a 3d character is shown in a ARSCNView. I am trying to get the character's head to track the ARCamera's point of view so they are always looking at the camera as the user moves to take a photo.

I've used Apple's chameleon demo, which adds a focus node that tracks the cameras point of view using SCNLookAtConstraint but I am getting strange behaviour. The head drops to the side and rotates as the ARCamera pans. If I add a SCNTransformConstraint to restrict the head movement to up/down/side-to-side, it stays vertical but then looks away and doesn't track.

I've tried picking the chameleon demo apart to see why mine is not working but after a few days I am stuck.

The code I am using is:

class Daisy: SCNScene, ARCharacter, CAAnimationDelegate {

    // Rig for animation
    private var contentRootNode: SCNNode! = SCNNode()
    private var geometryRoot: SCNNode!
    private var head: SCNNode!
    private var leftEye: SCNNode!
    private var rightEye: SCNNode!

    // Head tracking properties
    private var focusOfTheHead = SCNNode()
    private let focusNodeBasePosition = simd_float3(0, 0.1, 0.25)

    // State properties
    private var modelLoaded: Bool = false
    private var headIsMoving: Bool = false
    private var shouldTrackCamera: Bool = false



    /*
    * MARK: - Init methods
    */

    override init() {

        super.init()

        loadModel()
        setupSpecialNodes()
        setupConstraints()

    }


    /*
    * MARK: - Setup methods
    */

    func loadModel() {

        guard let virtualObjectScene = SCNScene(named: "daisy_3.dae", inDirectory: "art.scnassets") else {
            print("virtualObjectScene not intialised")
            return
        }

        let wrapper = SCNNode()

        for child in virtualObjectScene.rootNode.childNodes {
            wrapper.addChildNode(child)
        }

        self.rootNode.addChildNode(contentRootNode)

        contentRootNode.addChildNode(wrapper)

        hide()

        modelLoaded = true

    }


    private func setupSpecialNodes() {

        // Assign characters rig elements to nodes
        geometryRoot = self.rootNode.childNode(withName: "D_Rig", recursively: true)
        head = self.rootNode.childNode(withName: "D_RigFBXASC032Head", recursively: true)
        leftEye = self.rootNode.childNode(withName: "D_Eye_L", recursively: true)
        rightEye = self.rootNode.childNode(withName: "D_Eye_R", recursively: true)

        // Set up looking position nodes
        focusOfTheHead.simdPosition = focusNodeBasePosition

        geometryRoot.addChildNode(focusOfTheHead)

    }


    /* 
    * MARK: - Head animations
    */ 

    func updateForScene(_ scene: ARSCNView) {

        guard shouldTrackCamera, let pointOfView = scene.pointOfView else {
            print("Not going to updateForScene")
            return
        }

        followUserWithHead(to: pointOfView)

    }

    private func followUserWithHead(to pov: SCNNode) {

        guard !headIsMoving else { return }

        // Update the focus node to the point of views position
        let target = focusOfTheHead.simdConvertPosition(pov.simdWorldPosition, to: nil)

        // Slightly delay the head movement and the animate it to the new focus position
        DispatchQueue.main.asyncAfter(deadline: .now() + 0.2, execute: {
            let moveToTarget = SCNAction.move(to: SCNVector3(target.x, target.y, target.z), duration: 1.5)
            self.headIsMoving = true
            self.focusOfTheHead.runAction(moveToTarget, completionHandler: {
                self.headIsMoving = false
            })
        })

    }



    private func setupConstraints() {

        let headConstraint = SCNLookAtConstraint(target: focusOfTheHead)
        headConstraint.isGimbalLockEnabled = true

        let headRotationConstraint = SCNTransformConstraint(inWorldSpace: false) { (node, transform) -> SCNMatrix4 in

            // Only track the up/down and side to side movement
            var eulerX = node.presentation.eulerAngles.x
            var eulerZ = node.presentation.eulerAngles.z

            // Restrict the head movement so it doesn't rotate too far
            if eulerX < self.rad(-90) { eulerX = self.rad(-90) }
            if eulerX > self.rad(90) { eulerX = self.rad(90) }

            if eulerZ < self.rad(-30) { eulerZ = self.rad(-30) }
            if eulerZ > self.rad(30) { eulerZ = self.rad(30) }

            let tempNode = SCNNode()
            tempNode.transform = node.presentation.transform
            tempNode.eulerAngles = SCNVector3(eulerX, 0, eulerZ)
            return tempNode.transform

        }

        head?.constraints = [headConstraint, headRotationConstraint]

    }

    // Helper to convert degrees to radians
    private func rad(_ deg: Float) -> Float {
        return deg * Float.pi / 180
    }

}

The model in the Scene editor is:

Answer 1

I have solved the problem I was having. There were 2 issues:

1. The target in followUserWithHead should have converted the simdWorldPosition for it's parent and been convert from (not to)

   focusOfTheHead.parent!.simdConvertPosition(pov.simdWorldPosition, from: nil)

2. The local coordinates for the head node are incorrect. The z-axis should be the x-axis so when I got the focus the head movement tracking, the ear was always following the camera.

I didn't realise that the Debug View Hierarchy in Xcode will show the details of an SCNScene. This helped me to debug the scene and find where the nodes were tracking. You can export the scene as a dae and then load into SceneKit editor

Edit: I used localFront as mnuages suggested in the comments below, which got the tracking working in the correct direction. The head did occasionally moved about though. I have put this down to the animation that was running on the model trying to apply a transform that was then changed on the next update cycle. I decided to remove the tracking from the head and use the same approach to track the eyes only.