CODEWITHSUNDEEP
pythoncamera-calibrationpoint-cloudslidar
Eric
Eric

Reputation: 63

Projecting a point cloud onto the camera plane based on the projection matrix

I have a point cloud which I convert from .dat to .ply using open3d. The contents of the .ply file is an (nx3) matrix corresponding to the x, y, z points, as well as another (nx3) that corresponds to the RGB information. The total number of points are well over 2 million (the lidar was mounted on top of a vehicle). I also have a set of stereo cameras the were mounted along side the LiDAR (one left, one right), of which I only have the camera intrinsic parameters.

I am trying to replicate a formula found in several papers, which can be seen here, equations 2 & 3. It is originally found in the Kitti dataset paper, equation 8. Basically, they are projecting a point cloud based on the cameras projection with the following equation: where P is the projection matrix--containing the camera intrinsic parameters, R the rectifying rotation matrix of the reference camera, T_{cam}^{velo} the rigid boy transformation from lidar coordinates to camera coordinates, and T_{velo}^{imu}

I want to note that not all papers used the last parameter (T_{velo}^{imu}), and because I don't have the imu information I will omit that parameter.

While I only have the camera intrinsic parameters, I am able to extract the camera rotation and translation by way of the Essential matrix. Along with the data, I also have a file containing the yaw, pitch and roll (in degrees) of the camera and lidar at the time that the images where being taken. I know that I can extract a rotation matrix from these parameters, but I am not quite sure how to use them in this case, specifically to obtain the rigid body transformation from lidar to camera coordinates. I should also mention that I have the real world coordinates of the camera at the time each image was being taken (in x, y ,z coordinates).

Upvotes: 1

Views: 4172

Answers (1)

sowlosc
sowlosc

Reputation: 480

Here is the detailed answer to transform points between the lidar and the camera coordinates systems. I used the homogeneous representation of 3D transformations (4x4 matrices) because it is easier to manipulate, but it would be exactly the same if you used R and T instead. (the equation would just be longer)

enter image description here

Upvotes: 0

Related Questions