How to get rotation matrix from camera perspective

Question

I have a camera mounted on a post which is looking at a specific field. I know the exact real world position of the camera C and I know the exact size and real world position of the field P1, P2, P3, P4, W, H.

The problem is that the camera may rotate and I want to get exact rotation values (yaw, pitch, roll).

I can easily find all the corners and I used cv2.getPerspectiveTransform to get the perspective matrix H but I don't know how to extract rotation values/matrix from this.

I have a solution which gets some approximated values but it is very "heuristic" and seems way too complicated. I could also use cv2.solvePnP but it "tries" to estimate the position which is known so it also looks not optimal.

My common sense tells me that I should just decompose matrix H but I am just not sure how. I mean it is a 3x3 matrix and I should use 4x4. I could add some ones or zeros but I would really like to understand what I am doing instead of "just to have a working solution".

I would be really glad if someone could help me with this and give a simple explanation. I attach 3d projections for better understanding.

EDIT: I also did the calibration and have intrinsic params/matrix but for now I want to solve it on a "virtual" camera.

Thanks in advance!

Answer 1

Ok, I managed to find the solution - I used Grunert's Method as a reference.

So TLDR: Since I have the translation vector and 3 corresponding 3D-2D points so I can skip the first part of p3p problem (finding distances from camera to 3 points) and just focus on the second part which is rigid body transformation and getting R matrix from this equation:

It's easy to do using SVD decomposition and I used this as a reference.

@Micka thx again for your hints!