CODEWITHSUNDEEP
androidopengl-es-2.0arcore
snowrain
snowrain

Reputation: 442

ARCore screen coordinate to world coordinate openGL

I'm trying to place an anchor and display it where the user taps on the screen, which returns X and Y coordinates.

tapX = tap.getX();
tapY = tap.getY();

I want to use this information to create a matrix for my model. (i.e., put my 3D model where the user tapped)

Right now I tried:

 float sceneX = (tap.getX()/mSurfaceView.getMeasuredWidth())*2.0f - 1.0f;
float sceneY = (tap.getY()/mSurfaceView.getMeasuredHeight())*-2.0f + 1.0f; //if bottom is at -1. Otherwise same as X 
Pose temp = frame.getPose().compose(Pose.makeTranslation(sceneX, sceneY, -1.0f)).extractTranslation();

I'm just placing the 3D object 1 meter in front for now. I'm not getting proper location with this.

Is there a way to convert local coordinate to world coordinate?

Upvotes: 4

Views: 3763

Answers (1)

deeroh
deeroh

Reputation: 46

A tap on the screen doesn't have a one-to-one mapping to the real world. It gives you an infinite number of possible (x, y, z) positions that lie along a ray extending from the camera. If I tap in the center of the screen for example, that could correspond to an object floating a meter away from me, two meters away, on the floor, through the floor, etc.

So, you need some additional constraint to tell ARCore where along this ray to place your anchor. In the sample app, this is done by intersecting this ray with planes detected by ARCore. An ray intersecting a plane describes a single point, so you're set. This is how they do it in the sample app:

    MotionEvent tap = mQueuedSingleTaps.poll();
    if (tap != null && frame.getTrackingState() == TrackingState.TRACKING) {
        for (HitResult hit : frame.hitTest(tap)) {
            // Check if any plane was hit, and if it was hit inside the plane polygon.
            if (hit instanceof PlaneHitResult && ((PlaneHitResult) hit).isHitInPolygon()) {
                // Cap the number of objects created. This avoids overloading both the
                // rendering system and ARCore.
                if (mTouches.size() >= 16) {
                    mSession.removeAnchors(Arrays.asList(mTouches.get(0).getAnchor()));
                    mTouches.remove(0);
                }
                // Adding an Anchor tells ARCore that it should track this position in
                // space. This anchor will be used in PlaneAttachment to place the 3d model
                // in the correct position relative both to the world and to the plane.
                mTouches.add(new PlaneAttachment(
                    ((PlaneHitResult) hit).getPlane(),
                    mSession.addAnchor(hit.getHitPose())));

                // Hits are sorted by depth. Consider only closest hit on a plane.
                break;
            }
        }
    }

For your case in particular, I would take a look at the rendering code for the AR Drawing app. Their function GetWorldCoords() goes from screen coordinate to world coordinate assuming some known distance from the screen.

Upvotes: 1

Related Questions