CODEWITHSUNDEEP
opencvcoordinate-transformation
Samoo
Samoo

Reputation: 43

Finding the real world coordinates of an object from a camera

I am trying to find the coordinates of an object, which is detected from single camera, by using OpenCV. The camera will be mounted on the drone, looking through directly to the surface.

I have:

-Camera's coordinates from GPS sensor on the drone.

-Camera's height .

-Camera's intrinsic parameters.

3D Reconstruction formula

According to this formula, I need to find the extrinsic parameters to find the real world coordinates. I suppose to be use OpenCV’s solvePnP method to find extrinsic parameters. As I know, extrinsic parameters are about the camera location but my camera will be on the drone and the location will be change. Is the extrinsic parameters are constant just like the intrinsic parameters? Is there any other way to do this calculation?

Upvotes: 1

Views: 6954

Answers (1)

Henrik Larsen
Henrik Larsen

Reputation: 36

What you're trying to do is what is called monoplotting. In order to estimate XY real world coordinates from a single image you need to know the following:

  • X0, Y0, Z0 real world coordinates of your camera
  • Exterior orientation of your camera (Pitch, Roll and Yaw)
  • Interior orientation of your camera (focal length, principle point in x and y direction, radial and tangential distortion parameters)

So the XYZ of your camera and the exterior orientations are typically stored within the exif data of your image. This does however depend on the drone. There are some good python modules for extracting this information in the images. I use exifread.

The interior orientation can be more difficult to get, as you need to perform at camera calibration, which you can do with OpenCV. You should be able to use the tutorial Yunus linked in his comment. However there is a shortcut if you use photogrammetry software from pix4D, you can use their camera database which stores information about many different drones and their interior orientations. They are not perfect but should be alright for many use cases, see link.

When you have all of these parameters, you need to do the following:

  1. Undistort your images
  2. Create image coordinates of the points you wish to know the real world coordinate of, use undistorted image.
  3. Create rotation matrices with rotations along X,Y,Z axis and do matrix multiplikation on them (RXRYRZ)
  4. Apply collinearity equations

Regarding 1. you can use cv2.undistort for this, the link Yunus provided have a tutorial for this as well.

Regarding 3, I'm sure OpenCV probably can provide this matrix for you, but creating the function for your self is quite easy and good for understanding what is going on. See wikipedia: link It can be a little confusing which of the pitch, roll, yaw angles to use for which matrices. This all depends on how your cameras exterior coordinate system is. The typical convention is that pitch is around x, roll is around y and yaw is around z.

Regarding 4. The collinearity equations depends on the rotation matrices you created, see: link the term Z-Z0 just means the negative relative height above the object you try to find coordinates for. So if you do not have the relative height of your drone you need to know the height of your object and subtract the drones height from the objects height (you'll get a negative number).

I hope this helps you and have pointed you towards the right direction.

Upvotes: 1

Related Questions