Raycasting on Three.JS Object3D Meshes in Autodesk Forge Viewer

Question

I've been trying to perform raycasting on object that I've added in THREE.Scene, provided as viewer.impl.sceneAfter (I've followed this link as a tutorial for adding objects that support transparency).

As the document in the link specifies:

Unfortunately this has a side-effect: the native viewer selection mechanism has to deal with meshes that do not have the properties it expects.

It can be fixed but you likely have to edit the code of the viewer and load a custom version, see my fix below (viewer3D.js #L21962). Another option would be to create the custom geometry the same way the viewer does, so it could participate in the selection, but probably more work. If you have a better fix, I'm happy to hear it ...

At first, I've added regular THREE.Object3Ds which gave me an error when I performed a single click action:

wgs.js:1889 Uncaught TypeError: Cannot read property 'index' of undefined

Which I believe is, from viewer's meshRayCast function:

function meshRayCast( mesh, raycaster, intersects ) {

    init_three();

    var geometry = mesh.geometry;

    if (!geometry)
        return;

    var material = mesh.material;
    
    var side = material ? material.side : THREE.FrontSide;

    var attributes = geometry.attributes;

    inverseMatrix.getInverse( mesh.matrixWorld );
    ray.copy( raycaster.ray ).applyMatrix4( inverseMatrix );

    var a, b, c, i, j, il;
    var precision = raycaster.precision;
    var positions, stride, intersectionPoint, distance;

    if (attributes.index !== undefined) {   <----RIGHT HERE

From the source code, I've noticed that Autodesk Viewer expects THREE.BufferGeometry instead of regular geometry, so I've tested using Object3Ds created from BufferGeometries, which have gotten rid of error above on singleClick, but it still didn't support raycasting, even with my custom raycaster:

class SceneSelector {
  constructor(viewer){
    this.raycaster = new THREE.Raycaster();
    this.viewer= viewer;
  }

  selectByType(type,vectorXY){
    const camera = this.viewer.getCamera();
    const direction = new THREE.Vector3();
    const pointerDir = new THREE.Vector3();

    if(camera.isPerspective){
      direction.set(vectorXY.x,vectorXY.y,0.5);
      direction.unproject(camera);
      this.raycaster.set(camera.position,
                         direction.sub(camera.position).normalize());
    } else {
      direction.set(vectorXY.x,vectorXY.y,-1);
      direction.unproject(camera);
      this.raycaster.set(direction,
                         pointerDir.transformDirection(camera.matrixWorld));
    }
    const filteredObjects = this.viewer.impl.sceneAfter.children.filter(obj=>{
      return obj.type == type;
    });
    const intersects = this.raycaster.intersectObjects(filteredObjects,true);
    return intersects ? intersects[0] : null;
  }
};

I've tested various other viewer provided raycasting functions, provided by link, which didn't work for me.

(In summary) My question is:

1. Are there any ways that I can perform raycasting on objects added in THREE.SceneAfter?

2. It seems like the link suggests that I create custom geometry the same way the viewer does. Are there any reference on this?

Thanks in advance.

Answer 1

Raycasting on custom meshes works fine on my side, so please take a look at my RotateTool. It is using an overlay scene and I tried setting the transparency on the custom meshes which works fine. In the following code, if I replace the overlay by sceneAfter, it also works fine and I can detect mouse click on the gizmo using the raycaster:

createGizmo (center, euler, size, radius, color, range, axis) {

    var material = new GizmoMaterial({
      color: color
    })

    var subMaterial = new GizmoMaterial({
      color: color
    })

    var torusGizmo = new THREE.Mesh(
      new THREE.TorusGeometry(
        radius, size, 64, 64, range),
      material)

    var subTorus = new THREE.Mesh(
      new THREE.TorusGeometry(
        radius, size, 64, 64, 2 * Math.PI),
      subMaterial)

    subTorus.material.highlight(true)

    var transform = new THREE.Matrix4()

    var q = new THREE.Quaternion()

    q.setFromEuler(euler)

    var s = new THREE.Vector3(1, 1, 1)

    transform.compose(center, q, s)

    torusGizmo.applyMatrix(transform)

    subTorus.applyMatrix(transform)

    var plane = this.createBox(
      this.size * 100,
      this.size * 100,
      0.01)

    plane.applyMatrix(transform)

    subTorus.visible = false

    //this.viewer.impl.addOverlay(
    //  this.overlayScene, torusGizmo)
    //
    //this.viewer.impl.addOverlay(
    //  this.overlayScene, subTorus)

    this.viewer.impl.sceneAfter.add(torusGizmo)
    this.viewer.impl.sceneAfter.add(subTorus)

    torusGizmo.subGizmo = subTorus
    torusGizmo.plane = plane
    torusGizmo.axis = axis

    return torusGizmo
  }

Hope that helps