CODEWITHSUNDEEP
pythonopencvcomputer-visionhomography
fenaux
fenaux

Reputation: 47

compute homography between two images when camera is purely rotated

The Final question :

When camera make a pure Pan rotation how can I compute the homography H matrix between the the two images. H is a 3 x 3 matrix, H[0,0], H[0,1], H[1,0], H[1,1] seem easy to obtain from the rotations matrix but what about the others elements ?

Theoretical background

Given a camera matrix

K = np.diag([f, f, 1])

f beiing the focal length of the camera. The position of the camera center in world space coordinates

Tw = np.array([tx, ty, tz])

we build ITw

ITw = np.column_stack((np.eye(3), Tw))

we can compute the projective matrix

P = K @ Rot @ ITw

where Rot is a 3x3 rotation matrix Following for points belonging to the Z=0 plane in world coordinates we obtain an homography matrix

Hcalc = P[:,[0,1,3]]

Rot is defined as

from scipy.spatial.transform import Rotation as R
pan = 10
tilt = -30
roll = 4

pan *= np.pi / 180
tilt *= np.pi / 180
roll *= np.pi / 180

r_pan = R.from_euler('Z', pan).as_matrix()
r_tilt = R.from_euler('X', tilt).as_matrix()
r_roll = R.from_euler('Z', roll).as_matrix()

Rot = r_roll @ r_tilt @ r_pan

Illustration with an exemple

As exemple I take f = 400 and Tw= [10 , -5 , 6] and a collection of source points on the ground, I only give X and Y coordinates as Z = 0

src_pts = np.array([ 
       [ 0.  , -7.62],
       [ 0.  , -6.35],
       [ 0.  ,  0.  ],
       [ 0.  ,  1.27],
       [ 0.  ,  2.54],
       [ 0.  ,  3.81],
       [20.  , -7.62],
       [20.  , -6.35],
       [20.  ,  0.  ],
       [20.  ,  1.27],
       [20.  ,  2.54],
       [20.  ,  3.81]])

the I can computes Pts10 the image points on the screen for these points with Hcal and I can check that

h10, status = cv2.findHomography(src_pts.reshape(-1,1,2), pts10.reshape(-1,1,2))

leads to the same value Next I do exactly the same with pan = 15° to compute h15 and Pts15

When it becomes more difficult for me !

The homography from one image to the other is given by h, status = cv2.findHomography(pts10.reshape(-1,1,2), pts15.reshape(-1,1,2))

And I obtain

H
Out[187]: 
array([[ 9.97147963e-01, -7.56172873e-02,  1.73592176e+01],
       [ 7.54846993e-02,  9.98090883e-01,  1.87521001e+00],
       [-1.09070429e-04, -3.49361263e-06,  1.00000000e+00]])

Calling rot10 and rot15 the rotation matrix for pan = 10° and 15° respectively Then

rot15 @ rot10.T / (rot15 @ rot10.T)[2,2]
Out[191]: 
array([[ 0.99714794, -0.07561722,  0.04339806],
       [ 0.0754847 ,  0.9980909 ,  0.00468803],
       [-0.04362816, -0.00139744,  1.        ]])

It shows how to compute H[0,0], H[0,1], H[1,0], H[1,1]

Calling

rotNorm = rot15 @ rot10.T / (rot15 @ rot10.T)[2,2]

we have

H[i,j] = RotNorm[i,j] for i,j < 2

further

H[2,j] = RotNorm[2,j] / f for j < 2

but what about H[0,2] and H[1,2] ?

Thanks for reading me so far and I hope you can help me.

Upvotes: 0

Views: 33

Answers (1)

fenaux
fenaux

Reputation: 47

As often in math, trap is too think something is obvious ! True formula to go from one image to the other is

K @ rot15 @ rot10.T @ inv(K)

as K is diagonal I assumed that it has no effect but when I made the entire computation I found

K @ rot15 @ rot10.T @ np.linalg.inv(K)
Out[197]: 
array([[ 9.96199327e-01, -7.55452868e-02,  1.73427108e+01],
       [ 7.54128878e-02,  9.97141394e-01,  1.87342755e+00],
       [-1.08966647e-04, -3.49027100e-06,  9.99048675e-01]])

and that is H :))

Upvotes: 1

Related Questions