ARKit adding UIView xib file Swift 4

Question

I added a xib UIView file as material to my SCNBox when I detect my anchor but when I dismiss this viewcontroller the app freezes, this is my code:

var detectedDataAnchor: ARAnchor?
var myView = UIView()

override func viewDidLoad() {
  super.viewDidLoad()
  myView = (Bundle.main.loadNibNamed("ARViewOne", owner: nil, options: nil)![0] as? UIView)!
}



func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {

      if self.detectedDataAnchor?.identifier == anchor.identifier {
        let node = SCNNode()
        let box = SCNBox(width: 0.1, height: 0.1, length: 0.1,
                                 chamferRadius: 0.0)
        let imageMaterial = SCNMaterial()

        imageMaterial.diffuse.contents = myView
        box.materials = [imageMaterial]
        let cube = SCNNode(geometry: box)
        node.addChildNode(cube)
        return node

    }
   return nil
}

 @IBAction func back(_ sender: UIButton) {
     // here the app freezes
     navigationController?.popViewController(animated: true)
  }

And when back to my VC my app doesn't respond to any touch event

Answer 1

Okay, so the first thing first:

You cannot add a UIView as contents of a SCNMaterialProperty. Read up here.

I'm guessing here, but that's likely the cause of the freeze/crash you see, as the scene gets deinited when you pop the controller it's in and tries to clean up that material which is actually broken (it'll treat it as some other type).

Generally speaking the line

myView = (Bundle.main.loadNibNamed("ARViewOne", owner: nil, options: nil)![0] as? UIView)!

looks to me like you're a bit new to this (which is totally fine!). You're force-unwrapping and force-casting (in a weird way, as? UIView! is basically just as! UIView), which is always dangerous. Are you sure the first top level object in your xib is a UIView? Besides, the initially instantiated UIView (in var myView = UIView() is never used, why not going with an optional here (you could use an implicitly unwrapped one, though I am not a fan of this myself). How about doing it this way:

var myViewImage: UIImage?

override func viewDidLoad() {
    super.viewDidLoad()
    let myView = Bundle.main.loadNibNamed("ARViewOne", owner: nil, options: nil).first as? UIView
    myViewImage = myView?.asImage()
}

func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {

    guard let image = myViewImage, self.detectedDataAnchor?.identifier == anchor.identifier else { return nil }

    let node = SCNNode()
    let box = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0.0)
    let imageMaterial = SCNMaterial()
    imageMaterial.diffuse.contents = image
    box.materials = [imageMaterial]
    let cube = SCNNode(geometry: box)
    node.addChildNode(cube)
    return node
}

// this would be outside your controller class, i.e. the top-level of a swift file
extension UIView {
    func asImage() -> UIImage {
        let renderer = UIGraphicsImageRenderer(bounds: bounds)
        return renderer.image { rendererContext in
            layer.render(in: rendererContext.cgContext)
        }
    }
}

(The UIView extension was taken from here, so thanks Naveed J.)

This assumes that a) your xib is indeed correct and b) you just want the "view's looks", i.e. a kind of "screenshot" on the node. It'll be a still image, obviously, you can't get the entire behavior of a view (attached gesture recognizers and such) onto a node like this. For these things I'd suggest opening a new question (after you got this here done to your satisfaction).

Edit in response to Fabio's comment:

I forgot that renderer(_:nodeFor:) gets called on a different queue/thread. You should indeed generate the view image only on the main thread, as the warning "UIView.bounds must be used from main thread only" says. I changed the code above to reflect that. Keep in mind that swift is not inherently thread safe, above works since viewDidLoad() will be called (on the main thread) before the renderer starts sending renderer(_:nodeFor:) calls. Thus, you can be sure you won't get access conflicts when retrieving the image, but that means you should not modify said image somewhere on the main thread.

If you need to somehow change the image/create it on the fly there is an alternative. Personally I'd use the delegate's other method, renderer(_:didAdd:for:), instead of creating the node myself in renderer(_:nodeFor:). This method does not expect a return value, so it would look like this:

var myView: UIView?

override func viewDidLoad() {
    super.viewDidLoad()
    myView = Bundle.main.loadNibNamed("ARViewOne", owner: nil, options: nil).first as? UIView
}

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
    DispatchQueue.main.async {
        self.attachCustomNode(to: node, for: anchor)
    }
}

func attachCustomNode(to node: SCNNode, for anchor: ARAnchor) {
    guard let theView = myView, self.detectedDataAnchor?.identifier == anchor.identifier else { return }

    let box = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0.0)
    let imageMaterial = SCNMaterial()
    imageMaterial.diffuse.contents = theView.asImage()
    box.materials = [imageMaterial]
    let cube = SCNNode(geometry: box)
    node.addChildNode(cube)
    return node
}

By leaving out the renderer(_:nodeFor:) from your delegate, ARKit creates an empty node for you. Basically that's the same thing you do anyways in your renderer(_:nodeFor:). It then calls renderer(_:didAdd:for:), which doesn't expect a return value. So from there I "switch to the main queue" and add the necessary child.

I am pretty sure this works, IIRC SceneKit automatically batches the addition (i.e. what it actually does inside a node's addChildNode()) in its render thread. Thus, all the "prep work" above happens on the main thread, where you can safely use a view's bounds to create the image. The node gets set up and when you add it, SceneKit takes care of doing so properly on its render thread.

What you should not do is simply put a DispatchQueue.main.async inside your renderer(_:nodeFor:) method and change the material's diffuse content in there. That would mean you modify the added node from a different thread, but you can't be sure that works without conflicts. It might look like it work, and I don't know whether SceneKit somehow guards against that, but I wouldn't rely on that. It's bad style, imo anyways.

I hope this all is sufficient to make your project work. :)