CODEWITHSUNDEEP
pythonc++opencv
piro91
piro91

Reputation: 95

OpenCV find object's position with solvePnPRansac with not-corresponding points

I am trying to find object's position relative to camera position in real-world coordinates by tracking a known 2D LED pattern on the object.

I did camera calibration. I was able to sucessfully detect LEDs in the pattern and find their exact coordinates in the image frame. These points however do not correspond exactly 1-to-1 to the known coordinates in the pattern, they are in random order. The correspondence is important in functions like solvePnPRansac or findHomography, which would be my first choice to use.

How can I find the correspondence between these sets of points or maybe should I use some other function to calculate transformation just like solvePnPRansac does?

Upvotes: 0

Views: 1750

Answers (1)

BConic
BConic

Reputation: 8980

As you did not ask about the way to estimate the relative pose between your object and your camera, I will let this topic aside and focus on the way to find correspondences between each LED and their 2D projections.

In order to obtain a unique 1-to-1 correspondence set, the LED pattern you use should be unambiguous with respect to rotation. For example, you may use a regular NxN grid with the top-left cell containing an additional LED, or LEDs located on a circle with one extra LED underneath a single one, etc. Then, the method to find the correspondences depends on the pattern you chose.

In the case of the circle pattern, you could do the following:

  1. Estimate the center of gravity of the points
  2. Find the disambiguing point, which is the only one not lying on a circle, and define the closest of the other points as the first observed point
  3. Order the remaining points by increasing angle with respect to the center of gravity (i.e. clock-wise order)

In the case of the regular grid pattern, you could try the following:

  1. Find the four corners of the grid (those with min/max coordinates)
  2. Estimate the homography which transforms these four corners to the corners of a regular NxN square (with orthogonal angles)
  3. Transform the other points using this homography
  4. Find the disambiguing point, which is the only one for which X-floor(X) and Y-floor(Y) are close to 0.5, and define the closest of the four initial corners as the first observed point
  5. Order the remaining points by increasing angle with respect to the center of the grid and decreasing distance to the center of the grid

You could also study the algorithm used by the function findChessboardCorners (see calibinit.cppin the calib3D module), which uses a similar approach to order the detected corners.

Upvotes: 1

Related Questions