CODEWITHSUNDEEP
pythonpoint-cloudscamera-calibrationopen3d
Filipe Jorge
Filipe Jorge

Reputation: 13

Projecting a Texture Mask onto an existing 3D Mesh given the camera extrinsics

Given an image mask, I want to project the pixels onto a mesh in respect to the position and orientation of the camera and convert these pixels into a pointcloud. I have the intrinsic and extrinsic parameters of the camera in respect to the world, and the location of the mesh in world coordinates. I know the mapping from world coordinates to camera image is as follow:

imgpoint = Intrinsic * Extrinsic * worldpoint

So when I want to the opposite i do the inverse of the intrinsic and extrinsic matrices:

worldpoint= Intrinsic^(-1) * Extrinsic^(-1) * imgpoint

However, the idea that I had was to obtain two points from one pixel, with different depth values, to obtain a line and then look for the closest intersection for the mesh I want with the line, but I do not know how to properly generate a point away from the original camera plane. How can I find this extra point and/or am I complicating this problem?

Upvotes: 1

Views: 666

Answers (1)

wcochran
wcochran

Reputation: 10896

The top equation below shows how to project a point (x,y,z) onto a pixel (u,v); The extrinsic parameters are the 3x3 rotation matrix R and translation t. The intrinsic parameters are the focal distances f_x, f_y and principal point (c_x, c_y). The value alpha is the perspective foreshortening term that is divided out.

The bottom equation reverses the process by describing how to project a ray from the camera position through through the pixel (u,v) out into the scene as the parameter alpha varies from 0 to infinity.

enter image description here

Now we have converted the problem into a ray casting problem. Find the intersection of the ray with your mesh which is a standard computer graphics problem.

Upvotes: 0

Related Questions