CODEWITHSUNDEEP
c++opencvcomputer-visionlinear-algebra
Jukurrpa
Jukurrpa

Reputation: 4168

Use fundamental matrix to compute coordinates translation using OpenCV

I am trying to compute the coordinates correspondence of several points between two images. I have a group of points whose correspondences are known, I use them with OpenCV's findFundamentalMatrix() in order to find the fundamental matrix. I verified that x^T * F * x' = (0) for each point, and the result is always right or very close.

The thing is, now I'd like to use the coordinates of a point on the first image (y) and the fundamental matrix (F) in order to find the coordinates of the point on the second image (y'). I first thought about simply using the equation above, but given only the z of the y' point, there can be an infinity of solutions.

How else can I use the fundamental matrix to compute the translations ?

To be more clear: knowing the fundamental matrix "linking" two projections, how can I use it to translate the coordinates of any known point (a, b, 1) from the first projection to the second projection?

Considering that we know a, b and F in this equation: (a', b", 1)T * F * (a, b, 1) = (0)

I had made a simple drawing as an example: https://i.sstatic.net/l5yg4.jpg . The idea is to find the coordinates of the red dot (xq, yq) in projection 2, considering that we know its coordinates in projection 1 and the ones of all other points in both projections (and some other ones as the algorithm to find the fundamental matrix actually requires at least 8 points)

Another precision: in my example, known points are coplanar, but the researched point will not necessarily be.

I hope that made my problem more clear :)

Upvotes: 5

Views: 7815

Answers (3)

ychnh
ychnh

Reputation: 197

For some reason I could not add a comment due to a lack of reputation. I have been studying this field for about a month now and hopefully I can answer the many questions left unanswered that have also puzzled me when I was studying the topic.

@M2X A fundamental matrix is a mapping from a point in image plane 1 to a line in image plane 2. The lines are a special type of lines called epipolar lines and are formed by the intersection of the image plane and the plane constructed from the origin of the 2 cameras and the 3D point. So it is not possible to determine a point-point mapping using the Fundamental matrix unless you have some additional information or constraints.

@Jukurrpa A homography is a point to point mapping such that parallel lines map to parallel lines. One can prove that this mapping is linear, then since linear maps a equivalent to matrices, the homography can be defined by a matrix. A set of 3D points lying on a plane projected to the image plane maps parallel lines to parallel lines, so a homography will work in your case. Methods of estimating homograph from a given set of points is outlined in the book (multiple view geometry in computer vision). Given corresponding points in both images you can find homography by using iterative approaches (Gradient Descent) or closed form solutions (Singular Value Decomposition).

Upvotes: 2

Marina Tolkachov
Marina Tolkachov

Reputation: 51

Given:

Point correspondences a in image 1.

Goal:

Finding corresponding points b laying on the so called epipolar line L in image 2.

How?

    | x0 |       | x1 |
a = | y0 | , b = | y1 |
    | 1  |       | 1  |

L = F * a

    |F00 F01 F02|
F = |F10 F11 F12|
    |F20 F21 F22|

The following equation must be fulfilled to obtain b in image 2:

a' * F * b = 0.

Note: a' = transpose(a).

Upvotes: 2

fireant
fireant

Reputation: 14538

The fundamental matrix transforms points from one image to lines in the other. Could you elaborate more on

How else can I use the fundamental matrix to compute the translations?

please. Telling us what you want to achieve perhaps with an example would help too.

Edit: If you have calibrated the camera you can compute the essential matrix, E, from the fundamental matrix, F. E transforms a point in one image to a point in the other. But of course, the requirement is to have the internal matrix. If K is the internal matrix E=transpose(K)FK. The other method is to find the corresponding line for a point in the other image and then search along this line for the patch most similar in appearance to the patch surrounding the point in the first image. There are some other ways too but really need more information about the problem to tell which suits your case.

Edit 2: in the drawing you have got the points are coplanar. Hence, a homography maps the point positions between the two images, and there is no need to find the fundamental matrix. OpenCV has a function for estimating homographies, which needs only four points.

Upvotes: 3

Related Questions