ARKit create node from screen pan

Question

I am currently working on an app where the goal is for the user to be able to swipe their finger on the screen and see a line of 3D objects in the scene, similar to 3D drawing. I have my scene setup with the gesture recognizer but I do not know how to create a node in 3D space based on the pan on the screen. This is all I have right now for my gesture recognizer.

`@IBAction func panRecognized(_ sender: UIPanGestureRecognizer) {
    print("Panning")
    print(sender.location(in: self.view).x)
    print(sender.location(in: self.view).y)
}`

What I am wondering is how can I create a node in 3D space with those x and y coordinates while also using the orientation and position of the device? The goal is the object to be placed about half a meter away from the device.

Answer 1

Here are a couple of ways that you can achieve this, although please note that this is just a starting point, and isn't optimised in anyway:

The first thing you would do is create a UIPanGestureRecognizer in viewDidLoad for example e.g:

let panToDrawGesture = UIPanGestureRecognizer(target: self, action: #selector(createNodesFromPan(_:)))
self.view.addGestureRecognizer(panToDrawGesture)

To make it a bit more fun, also add an array of [UIColor] e.g:

let colours: [UIColor] = [.red, .green, .cyan, .orange, .purple]

Then to draw using your gestureRecognizer you can do something like this:

///Creates An SCNNode At The Touch Location Of The Gesture Recognizer
@objc func createNodesFromPan(_ gesture: UIPanGestureRecognizer){

    //1. Get The Current Touch Location
    let currentTouchPoint = gesture.location(in: self.augmentedRealityView)

    //2. Perform An ARHitTest For Detected Feature Points
    guard let featurePointHitTest = self.augmentedRealityView.hitTest(currentTouchPoint, types: .featurePoint).first else { return }

    //3. Get The World Coordinates
    let worldCoordinates = featurePointHitTest.worldTransform

    //4. Create An SCNNode With An SCNSphere Geeomtery
    let sphereNode = SCNNode()
    let sphereNodeGeometry = SCNSphere(radius: 0.005)

    //5. Generate A Random Colour For The Node's Geometry
    let randomColour = colours[Int(arc4random_uniform(UInt32(colours.count)))]
    sphereNodeGeometry.firstMaterial?.diffuse.contents = randomColour
    sphereNode.geometry = sphereNodeGeometry

    //6. Position & Add It To The Scene Hierachy
    sphereNode.position = SCNVector3(worldCoordinates.columns.3.x,  worldCoordinates.columns.3.y,  worldCoordinates.columns.3.z)
    self.augmentedRealityView.scene.rootNode.addChildNode(sphereNode)
}

Alternatively instead of using a gestureRecognizer you could use touches to perform you drawing:

override func touchesMoved(_ touches: Set<UITouch>, with event: UIEvent?) {

    //1. Get The Current Touch Location
    guard let currentTouchPoint = touches.first?.location(in: self.augmentedRealityView),
        //2. Perform An ARHitTest For Detected Feature Points
        let featurePointHitTest = self.augmentedRealityView.hitTest(currentTouchPoint, types: .featurePoint).first else { return }

    //3. Get The World Coordinates
    let worldCoordinates = featurePointHitTest.worldTransform

    //4. Create An SCNNode With An SCNSphere Geeomtery
    let sphereNode = SCNNode()
    let sphereNodeGeometry = SCNSphere(radius: 0.005)

    //5. Generate A Random Colour For The Node's Geometry
    let randomColour = colours[Int(arc4random_uniform(UInt32(colours.count)))]
    sphereNodeGeometry.firstMaterial?.diffuse.contents = randomColour
    sphereNode.geometry = sphereNodeGeometry

    //6. Position & Add It To The Scene Hierachy
    sphereNode.position = SCNVector3(worldCoordinates.columns.3.x,  worldCoordinates.columns.3.y,  worldCoordinates.columns.3.z)
    self.augmentedRealityView.scene.rootNode.addChildNode(sphereNode)

}

In my example augmentedRealityView refers to an ARSCNView e.g:

@IBOutlet weak var augmentedRealityView: ARSCNView!

Hope it helps...

Update:

If you want to draw at a set distance, and only using the position of the ARCamera you can do something like this using the ARSessionDelegate(and if you want your drawing to be say 1m away from the camera then change the Z value of the sphereNode.position in part 3):

func session(_ session: ARSession, didUpdate frame: ARFrame) {

    //1. Create An SCNNode With An SCNSphere Geeomtery
    let sphereNode = SCNNode()
    let sphereNodeGeometry = SCNSphere(radius: 0.01)

    //2. Generate A Random Colour For The Node's Geometry
    let randomColour = colours[Int(arc4random_uniform(UInt32(colours.count)))]
    sphereNodeGeometry.firstMaterial?.diffuse.contents = randomColour
    sphereNode.geometry = sphereNodeGeometry

    //3. Position & Add It To The Scene Hierachy
    sphereNode.position = SCNVector3(0,  0, -0.5)
    updatePositionAndOrientationOf(sphereNode, withPosition: sphereNode.position, relativeTo: self.augmentedRealityView.pointOfView!)
    self.augmentedRealityView.scene.rootNode.addChildNode(sphereNode)
}


/// Updates The Position Of An SCNNode In Relation To The Camera Node
///
/// - Parameters:
///   - node: SCNNode
///   - position: SCNVector3
///   - referenceNode: SCNNode
func updatePositionAndOrientationOf(_ node: SCNNode, withPosition position: SCNVector3, relativeTo referenceNode: SCNNode) {

    /* Full Credit To Pablo
    https://stackoverflow.com/questions/42029347/position-a-scenekit-object-in-front-of-scncameras-current-orientation/42030679
    */

    let referenceNodeTransform = matrix_float4x4(referenceNode.transform)

    // Create A Translation Matrix With The Desired Position
    var translationMatrix = matrix_identity_float4x4
    translationMatrix.columns.3.x = position.x
    translationMatrix.columns.3.y = position.y
    translationMatrix.columns.3.z = position.z

    // Multiply The Configured Translation With The ReferenceNode's Transform
    let updatedTransform = matrix_multiply(referenceNodeTransform, translationMatrix)
    node.transform = SCNMatrix4(updatedTransform)
}