2D coords to 3D coords resolution and camera matrix

Question

So I have a full HD webcam (LG AN VC500), looking down at a chessboard, with each square edge at 25mm. I am trying to estimate the 3 dimensional coordinates of the camera, relative to the chessboard.

I use openCV and C++ and the following basic procedure:

• calibrate camera -> to get camera matrix and distortion coefficients
• findChessboardCorners -> to get imagePoints
• SolvePnP -> to get rvec and tvec
• rodrigues -> rvec to RMatrix
• transpose RMatrix
• (-RMatrix * tvec) -> to get TVector

• pose = [RMatrix TVector(0) ... TVector(1) ... TVector(2) 0, 0, 0, 1 ]

• Plot in openGL^

^ I am aware of the axes difference between GL and CV, the main focus is on the pose estimation, not the representation.

As far as I know, this method should work. But my translations are not giving me anywhere near the correct values. Not only in correct unit of distance, but the x, y and z values don't even make sense relative to each other. According to my results the origin of the chessboard is in some arbitrary block, not an outer corner.

I suspect the issue might be the fact that my camera is calibrated at its streaming resolution 1280 x 720, therefore giving me a camera matrix according to that resolution, while openCV only deals with a 640 x 480 image.

My questions are:

1. Is the procedure I have followed correct? If not, what have I done wrong, and how do I get my translations in millimeters?
2. Does the resolution discrepancy make a difference? Should I modify specific code, or camera matrices.

I have spent a significant amount of time trying to figure this out, most of which spent on stack overflow. My deadline is now becoming real. Any help is appreciated, and I will give more information if needed. Thanks.

Answer 1

The procedure that I mentioned above is correct. The results were also correct, but I misinterpreted them. The result is the translation coordinates of the camera center perpendicular to the chess board, relative to the chessboard origin. The result is also in the units that the camera was calibrated. If calibrated in millimeters, then the translation vector given by solvepnp will also be in millimeters.

I thought it was the center principal point line w.r.t the chessboard origin.

As for the resolution discrepancy, if your camera is set (in openCV code) to a specific resolution that is compatible with the camera (check this compatibility) then openCV will handle the images at that size.

Answer 2

SolvePnP uses the camera matrix which includes the image center (from the camera calibration) if you are using it at any other resolution you will have to adjust the camera matrix.

Details will depend on whether you are taking a 640x480 window out of the image (in which case just adjust the origin) or rescaling the image (in which case you also need to change the focal length)