CODEWITHSUNDEEP
opencvcomputer-vision
Humam Helfawi
Humam Helfawi

Reputation: 20274

Composing multiple Essential matrices

I have computed the Essential matrices between Frame [a,b], [b,c] and [c,d]. I have now E_ab, E_bc and E_cd. Is it possible to compute E_ad directly without matching? I am thinking of sth like 3D transformation composing where:

T04 = T01 * T12 * T23 * T34

The solution that I found is to decompose each one it into R|T and construct T_4*4 from R|T and then use the previouse notation of composing T matrices and then convert the final R|T to E. However, I was wondering if there is a direct way without decomposing for two reasons:

So, How can I combine two (or more) essential matrices directly?

Upvotes: 2

Views: 264

Answers (1)

Ash
Ash

Reputation: 4718

Before answering the original question, I think it might be important to discuss this point from your comment:

Ultimate goal is Bundle Adjustment on the Essential Matrices directly instead of on RT. I am using Single Calibrated Camera

It might be that I'm missing critical details about your project, but there seems to be a few issues with that method:

  1. Minimal representation. Usually, it's better (for both performance and accuracy reasons) to use minimal parametrisations in your estimations. The Essential matrix has six degrees of freedom (which can be seen from its decomposition as [T]_xR). If you estimate your Essential directly, you're optimizing eight parameters instead of six.

  2. Less constrained problem. The Essential matrix imposes coplanarity constraints and doesn't penalize some wrong configurations that would be penalized by reprojection errors that are used in usual bundle adjustment. If you consider two cameras C_1 and C_2 such that a point X_2 expressed in C_2 can be expressed in C_1 via RX_2+T, and that we denote the corresponding point in C_1 as X_1, then the constraint X_1.transpose()*E*X_2=0 imposes that the three vectors T, X_1 and RX_2+T remain coplanar. So, the point X_2 is free to move as long as it reprojects to the epiploar line, even if its ray diverges from X_1:

    enter image description here

    As you can see in the figure, the red and green points would respect the Essential matrix constraint, however they would have different reprojection errors.

  3. Time complexity. Computing Essentials is really slow compared to alternatives, and so is the recovery of rotation and translations from Essentials, compared to the inverse problem.

Now to come back to the initial problem from the question, if you consider the three cameras C_1, C_2, C_3 and respectively denote their poses as Identity, R_1, R_2, and note the Essentials between C_1 and C_2 as E_1 and the one between C_2 and C_3 as E_2, then you can easily see that the Essential from C_3 to C_1, that I'll note E_3, will be given by

E_3=[R_1T_2+T_1]_x R_1R_2

which with a little bit of reorganization leads to 

E_3=R_1E_2 + E_1R_2

This relation which establishes the link between the three Essential matrices seems to indicate that you still will need to extract R_1 and R_2 from E_1 and E_2.

Edit: I derived the relationship above like this: assume y is a 3x1 vector, then

E_3y = [R_1T_2+T_1]_x R_1R_2y
     = (R_1T_2+T_1) x (R_1R_2y)
     = R_1T_2 x R_1R_2y + T_1 x R_1R_2y #for the next line, note that crossproduct is invariant under rotation
     = R_1(T_2 x R_2 y) + T_1 x R_1R_2 y
     = R_1[T_2]_x R_2 y + [T_1]x_x R_1 R_2 y
     = R_1 E_2 y + E_1 R_2 y
     = (R_1E_2 + E_1R_2) y

Since this holds for any arbitrary y, this means that E_3=R_1E_2+E_1R_2.

Upvotes: 1

Related Questions