CODEWITHSUNDEEP
graphics3dhololenspoint-cloud-librarytexture-mapping
donaldc
donaldc

Reputation: 1

Camera projection onto spatial map with Hololens 2

I am trying to project the recorded image (with some modification of the color) on to the spatial map generated from the 3D view of Hololens 2 device portal. I recorded the image with the corresponding CameraToWorld matrix and Projection Matrix

for example

Projection Matrix:
[1.53862 0.00000 0.00729 0.00000],
[0.00000 2.73525 -0.05193 0.00000],
[0.00000 0.00000 -1.00000 0.00000],
[0.00000 0.00000 -1.00000 0.00000]

CameraToWorld Matrix:
[0.99903 0.03822 -0.02210 0.01095],
[-0.03472 0.98934 0.14145 0.10689],
[-0.02727 0.14054 -0.98970 0.16761],
[0.00000 0.00000 0.00000 1.00000]

)

When I take the CameraToWorld Matrix and apply it to PCL texturemapping, I found that the image is projected to the opposite side and upside down. Hence, I add a x-axis rotation with 180degree.

I have got the following result where the position is not at the correct position

The red square should be the correct position of monitor

I have the following questions

  1. Why there is a rotation offset? What is the correct way to apply the CameraToWorld matrix to the PCL texture-mapping function?

  2. I haven't use the Projection Matrix, am I missing anything during the projection process?

Upvotes: -1

Views: 426

Answers (1)

Sarah Zhang - MSFT
Sarah Zhang - MSFT

Reputation: 36

Do you use the trimesh in the python project? If so, the coordinate system of the scene should be the left-handed coordinate system, and the coordination system of the Spatial mapping is the right-handed one, so there would be a rotation offset. To get the correct location of the spatial mapping, you need to convert the matrix to the corresponding left-handed coordination matrix.

The projection matrix is designed for the transform from the world coordination to the camera coordination. If you want to add the textures to 3D models, you needn’t use it.

Upvotes: 0

Related Questions