CODEWITHSUNDEEP
opencv3d2dtransform
never_ever
never_ever

Reputation: 185

OpenCV - get 3D coordinates from 2D

I saw that lots of people ask similar questions, but I can't exactly match any answer to my problem

I have marker and I get (x,y) of it in image coordinates. I can get transformation matrix and rotation matrix. Now how can I get (x,y,z) coordinates?

(if anyone could advise me how to solve my problem or redirect I would be grateful).

EDIT: I use Alvar library, where I get marker coordinates, and I can get also pose of it (so I have also rotation matrix 3x3, and translation matrix 3x4 or 4x4)

EDIT 2: I searched more and I study the method that allow me to acquire the rotation and translation matrices. Well I suposed that I can get two matrices depends on input arguments. But If I set as a argument 3x3 matrix I will have rotation matrix only, but if I put there 3x4 or 4x4 matrix I will get:

Upvotes: 2

Views: 10251

Answers (1)

Kornel
Kornel

Reputation: 5364

Say you have a image point in pixel unit on the image plane: cv::Vec2d imagePoint(u, v);

First of all, you should transform this point in the camera coordinate system. Assuming that the intrinsics parameters (camera matrix and lens distortion coefficients) of your camera is known, you can compute the ideal (u', v') coordinates of your observed imagePoint:

u'' = (u - cx)/fx
v'' = (v - cy)/fy
(u', v') = undistort(u'', v'', distCoeffs)

Where cx and cy are the coordinates of principal points that is usually around the image center, furthermore fx and fy are the focal lengths in pixel units (you can get them from the camera matrix). And distCoeffs contains the radial/tangential distortion coefficients, which are also the results of cv::calibrateCamera(...).

Anyway, you shouldn't compute (u', v') manually, because cv::undistortPoints(...) does this by calling the function with no or the default R and P parameters.

The point of imagePoint in the camera coordinate system is the following:

std::vector<cv::Vec2d> imagePts;
std::vector<cv::Vec2d> idealPts;
imagePts.push_back(imagePoint);

cv::undistortPoints(imagePts, idealPts, cameraMatrix, distCoeffs);

const double lambda = 1.0;
cv::Mat cameraPt(3, 1, CV_64F);
cameraPt.at<double>(0) = idealPts[0][0] * lambda;
cameraPt.at<double>(1) = idealPts[1][1] * lambda;
cameraPt.at<double>(2) = lambda;

At this point, you need the camera to world transformation matrix to express cameraPt in the world coordinate system:

cv::Mat camToWorld = cv::Mat::eye(4, 4, CV_64FC1);
// Fill camToWorld with [R^T|-R^T*t]
// ...

cameraPt.push_back(1.0);
cv::Mat worldPt = camToWorld * cameraPt;

So far, worldPt defines a ray in the world coordinate system corresponding to imagePoint. That is each points of the ray/line can be projected to the same imagePoint, so you have infinite number of points in the world which belong to the same imagePoint. But, for example with Möller–Trumbore ray-triangle intersection algorithm you can determine one 3-D point on a plane of the world.

Upvotes: 10

Related Questions