How to tap and move scene nodes in ARKit

Question

I'm currently trying to build an AR Chess app and I'm having trouble getting the movement of the pieces working.

I would like to be able to tap on a chess piece, then the legal moves it can make on the chess board will be highlighted and it will move to whichever square the user tapped on.

Pic of the chess board design and nodes: https://gyazo.com/2a88f9cda3f127301ed9b4a44f8be047

What I would like to implement: https://i.sstatic.net/oAr5K.jpg

Would greatly appreciate any suggestions on how to get this working.

Thanks!

ViewController Code:

import UIKit
import SceneKit
import ARKit

class ViewController: UIViewController, ARSCNViewDelegate {

    @IBOutlet var sceneView: ARSCNView!

    override func viewDidLoad() {
        super.viewDidLoad()

        // Set the view's delegate
        sceneView.delegate = self

        // Show statistics such as fps and timing information
        sceneView.showsStatistics = true

        // Add lighting to the scene
        sceneView.autoenablesDefaultLighting = true
    }

    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)

        // Create a session configuration to track an external image
        let configuration = ARImageTrackingConfiguration()

        // Image detection
        // Reference which group to find the image to detect in the Assets folder e.g. "Detection Card"
        if let imageDetect = ARReferenceImage.referenceImages(inGroupNamed: "Detection Card", bundle: Bundle.main) {
            // Sets image tracking properties to the image in the referenced group
            configuration.trackingImages = imageDetect
            // Amount of images to be tracked
            configuration.maximumNumberOfTrackedImages = 1
        }

        // Run the view's session
        sceneView.session.run(configuration)
    }

    // Run when horizontal surface is detected and display 3D object onto image
    // ARAnchor - tells a certain point in world space is relevant to your app, makes virtual content appear "attached" to some real-world point of interest
    func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode {
        // Creates 3D object
        let obj = SCNNode()
        // Check if image detected through camera is an ARImageAnchor - which contains position and orientation data about the image detected in the session
        if let imageAnchor = anchor as? ARImageAnchor {
            // Set dimensions of the horizontal plane to be displayed onto the image to be the same as the image uploaded
            let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
            // Display mild transparent layer onto detected image
            // This is to ensure image detection works by display a faint layer on the image
            plane.firstMaterial?.diffuse.contents = UIColor(white: 1.0, alpha: 0.2)
            // Set geometry shape of the plane
            let planeNode = SCNNode(geometry: plane)
            // Flip vertical plane to horizontal plane
            planeNode.eulerAngles.x = -Float.pi / 2
            obj.addChildNode(planeNode)

            // Initialise chess scene
            if let chessBoardSCN = SCNScene(named: "art.scnassets/chess.scn") {
                // If there is a first in the scene file
                if let chessNodes = chessBoardSCN.rootNode.childNodes.first {
                    // Displays chessboard upright
                    chessNodes.eulerAngles.x = Float.pi / 2
                    // Adds chessboard to the overall 3D scene
                    obj.addChildNode(chessNodes)
                }
            }

        }

        return obj

    }

    override func viewWillDisappear(_ animated: Bool) {
        super.viewWillDisappear(animated)

        // Pause the view's session
        sceneView.session.pause()
    }

}

Answer 1

First, you need to add a gesture recognizer for tap into your viewDidLoad, like this:

 let tapGesture = UITapGestureRecognizer(target: self, action: #selector(handleTap(_:)))
 myScnView.addGestureRecognizer(tapGesture)

Then realize the handler function:

    @objc
    func handleTap(_ gestureRecognize: UIGestureRecognizer) {
       // HERE YOU NEED TO DETECT THE TAP
       // check what nodes are tapped

        let location = gestureRecognize.location(in: myScnView)
        let hitResults = myScnView.hitTest(location, options: [:])

        // check that we clicked on at least one object
        if hitResults.count > 0 {
            // retrieved the first clicked object
            let tappedPiece = hitResults[0].node

            // HERE YOU CAN SHOW POSSIBLE MOVES
            //Ex. showPossibleMoves(for: tappedPiece) 
        }

    }

Now, to show the possible moves, you need to identify all quadrants and your node position on the chessboard.

To do this, you can assign a name or a number, or a combination of letter and number, or moreover a combination of numbers. (I suggest combination of number, like row 1 column 1, like a matrix).

let's take my suggestion, so you can name each quadrant 1.1 1.2 ... 2.1 2.2 and so on.

Now, to detect where your piece is, you can check contact with the PhysicsContactDelegate.

Now you have the tappedPiece and the place where it is, so you have to define the rule for the pieces, for example:

let rules = ["tower":"cross"] //add the others

N.B You can choose what you want to define the rules.

Let's take my suggestion for good, now you should create the function to highlight:

 func highlight(quadrant: SCNNode){
   quadrant.geometry?.firstMaterial?.emission.contents = UIColor.yellow
 }

Finally the showPossibleMoves(for: tappedPiece) could be something this:

func showPossibleMoves(for piece: SCNNode){

let pieceType = piece.name //You have to give the name as you did into your rules variable
 //ex. if you have rules like ["tower":"cross"] you have to set all towers name to "tower"

let rule = rules[pieceType]

switch rule{

case "cross":

      //you have to highlight all nodes on the right, left, above and bottom
      // you can achieve this by selecting the start point and increase it
      //assuming you named your quadrants like 1.1 1.2 or 11 12 13 ecc...
      let startRow = Int(startQuadrant.name.first)
      let startColumn = Int(startQuadrant.name.last)

      //Now loop the highlight on right
      for column in startColumn+1...MAX_COLUMN-1{
         let quadrant = myScnView.scene.rootNode.childNode(withName:"\(startRow).\(column)" , recursively: true)
         // call highlight function
          highlight(quadrant: quadrant)
      }

      //Now loop for above quadrants
      for row in startRow+1...MAX_ROW-1{
         let quadrant = myScnView.scene.rootNode.childNode(withName:"\(row).\(startColumn)" , recursively: true)
         // call highlight function
          highlight(quadrant: quadrant)
      }

      //DO THE SAME FOR ALL DIRECTIONS
}

// ADD ALL CASES, like bishop movements "diagonals" and so on

}

NOTE: In the handlerTap function you have to check what you're tapping, for example, to check if you're tapping on a quadrant after selecting a piece (you want to move you're piece) you can check a boolean value and the name of the selected node

  //assuming you have set the boolean value after selecting a piece

  if pieceSelected && node.name != "tower"{

      //HERE YOU CAN MOVE YOUR PIECE

  }

Answer 2

You will need to add gestures on to your view and use the ARSceneViews hitTest method to detect what the gesture is touching in your scene. You can then update the positions based on the movement from the gestures.

Here is a question that deals with roughly the same requirement of dragging nodes around.

Placing, Dragging and Removing SCNNodes in ARKit