Dynamically place object in front of the user with ArCore

Question

I'm using ArCore to build a simple navigation app for Android (in Java). I started building my app on the sceneform demo from Google (https://github.com/google-ar/sceneform-android-sdk/tree/master/samples/hellosceneform).

Currently the app is scanning the environment for anchor points. The user can place an object on the anchor points by tapping on them. Is it possible to use these found anchor points to automatically place for example an arrow for navigation in front of the user? The arrow should maybe be 3m-5m in front of the user and move if the user gets to close.

I know how to look through the anchor points and always delete the last anchor. But I'm unsure how to place the next object automatically in front of the user. Is there a way to do that?

Edit: Is there no way to do that? Maybe read out the position from the anchor and look for with the needed distance?

Answer 1

Here is the code snippet to place the object in center of the screen

    override fun onUpdate(frameTime: FrameTime?) {
    arFragment?.let { fragment ->
        fragment.arSceneView?.let { sceneView ->
            sceneView.arFrame?.let { frame ->
                if (!placed) {
                    val trackable = frame.getUpdatedTrackables(Plane::class.java).iterator()
                    if (trackable.hasNext()) {
                        val plane = trackable.next() as Plane
                        if (plane.trackingState == TrackingState.TRACKING) {
                            fragment.planeDiscoveryController?.hide()
                            val hitTest =
                                frame.hitTest(frame.screenCenter().x, frame.screenCenter().y)
                            val hitTestIterator = hitTest.iterator()
                            if (hitTestIterator.hasNext()) {
                                val hitResult = hitTestIterator.next()
                                val modelAnchor = plane.createAnchor(hitResult.hitPose)
                                val anchorNode = AnchorNode(modelAnchor)
                                anchorNode.setParent(sceneView.scene)

                                val transformableNode =
                                    TransformableNode(fragment.transformationSystem)
                                transformableNode.setParent(anchorNode)
                                transformableNode.renderable = this@ARActivity.modelRenderable

                                transformableNode.worldPosition = Vector3(
                                    modelAnchor.pose.tx(),
                                    modelAnchor.pose.compose(
                                        Pose.makeTranslation(
                                            0f,
                                            0.05f,
                                            0f
                                        )
                                    ).ty(),
                                    modelAnchor.pose.tz()
                                )
                                placed = true
                            }
                        }
                    }
                }
            }
        }
    }
}

private fun Frame.screenCenter(): Vector3 {
    val vw = findViewById<View>(android.R.id.content)
    return Vector3(vw.width / 2f, vw.height / 2f, 0f)
}

Answer 2

I am hoping this is what you are looking for. This will render the arrow in front of the phone once the AR scene starts tracking feature points.

//Add a boolean value to ensure only 1 arrow is rendered.

 boolean placed = false;

//Add an on update listener in the onCreate function, as it takes a sec or two for the application to get the Pose of the camera and the app might crash if you try to make the anchor from a null Pose.

arFragment.getArSceneView().getScene().addOnUpdateListener(this::onUpdateFrame);

//You can get the pose of the camera to make an anchor in front of the camera accordingly in the update function.

private void onUpdateFrame(FrameTime frameTime) {

    Frame frame = arFragment.getArSceneView().getArFrame();

    // If there is no frame, just return.
    if (frame == null) {
        return;
    }

  //Making sure ARCore is tracking some feature points, makes the augmentation little stable. 
  if(frame.getCamera().getTrackingState()==TrackingState.TRACKING && !placed) {

      Pose pos = frame.getCamera().getPose().compose(Pose.makeTranslation(0, 0, -0.3f));
      Anchor anchor = arFragment.getArSceneView().getSession().createAnchor(pos);
      AnchorNode anchorNode = new AnchorNode(anchor);
      anchorNode.setParent(arFragment.getArSceneView().getScene());

  // Create the arrow node and add it to the anchor.
  Node arrow = new Node();
  arrow.setParent(anchorNode);
  arrow.setRenderable(arrowRenderable);
  placed = true; //to place the arrow just once.

  }

}

You will have to make changes to transformation of the arrow to set it exactly in the centre in front of the camera and in the right orientation. I hope this gets you started. Cheers !