CODEWITHSUNDEEP
pythonopencvpose-estimation
HighVoltage
HighVoltage

Reputation: 742

Relative pose estimation using essential matrix: Wrong R and T

I am trying to use the essential matrix method in opencv to obtain the R and t of one camera pose with respect to another. The procedure I am following is:

  1. Mark features using SIFT
  2. Match features using FLANN matching
  3. Compute fundamental matrix.
  4. Compute essential matrix
  5. Perform SVD to obtain U, W, Vt
  6. Check to see which combination of R and t is correct according to whether or not the normalized/homogenized points are in front of the camera.

For a simple check, I tested this with an image pair but of the same image twice (so that neither the camera nor the image points have moved), hence the translation vector should be null and the rotation should be identity. But the output of the program ends up being wrong. 

The fundamental matrix is
[[  3.59955121e-17  -5.77350269e-01   2.88675135e-01]
 [  5.77350269e-01   5.55111512e-17   2.88675135e-01]
 [ -2.88675135e-01  -2.88675135e-01   0.00000000e+00]]

Fundamental matrix error check: 0.000000

The essential matrix is
[[  4.51463713e-10  -7.25229650e+06  -2.37367600e+06]
 [  7.25229650e+06   6.98357978e-10   4.27847619e+06]
 [  2.37367600e+06  -4.27847619e+06  -1.33013600e-10]]

Translation matrix is
[-0.48905495 -0.2713251   0.82898007]

Rotation matrix is
[[ 0.52165052 -0.26538577  0.8108336 ]
 [-0.26538577  0.85276538  0.4498462 ]
 [ 0.8108336   0.4498462  -0.3744159 ]]
Roll: -26.965168, Pitch: 129.775110, Yaw: -54.179055

I also used this code with a pair of cameras, displaced by a certain distance in X: but the Euler angles and translation I obtain using this technique (there, I consider two camera matrices instead of one) are still wrong. The translation vector tells me that I've moved in both X and Z, and the rotation matrix is not accurate. I am confused as to what might be going wrong here. Any suggestions would be very helpful. Thank you!

EDIT: My code can be viewed here

Upvotes: 0

Views: 2014

Answers (1)

troymyname00
troymyname00

Reputation: 702

I think before proceeding with matching features, you need to un-distort the images using the camera matrices and distortion coefficients. I know this is too late, but I hope this helps others.

Upvotes: 0

Related Questions