How do I access the rotation and translation vectors after camera calibration in emgu CV?

Question

The goal of a camera calibration is to find the intrinsic and extrinsic parameters:

1. The intrinsic ones are those that describe the camera itself (focal length, distortion, etc.) I get values for those, no problem.
2. The extrinsic parameters are basically the position of the camera. When I try to access those I get an AccessViolationException.

One way to perform such calibration is to

1. take an image of a calibration target with known corners
2. find those corners in the image
3. from the correspondence between 3D and 2D points, find the matrix that transforms one into the other
4. that matrix consists of the intrinsic and extrinsic parameters.

The call to the calibration function looks like this:

Mat[] rotationVectors = new Mat[1];
Mat[] translationVectors = new Mat[1];

double error = CvInvoke.CalibrateCamera(realCorners,
                                        detectedCorners,
                                        calibrationImages[0].Image.Size,
                                        cameraMatrix,
                                        distortionCoefficients,
                                        0,
                                        new MCvTermCriteria(30, 0.1),
                                        out rotationVectors,
                                        out translationVectors);

Console.WriteLine(rotationVectors[0].Size); // AccessViolationException

• I only use one image here, but I have the same problem when using more images (30) Different calibration images would yield different results for translation-/rotationVector anyway, which makes me doubt that using only 1 image is a problem.
• The detection of points works and drawing them into the original image gives reasonabel results.
• Both cameraMatrix and distortionCoefficients can be accessed and contain values. (I tried to only post the relevant parts of the code)
• I use emgu version 3.0.0.2157

Why do I get an AccessViolationException on the rotationVectors and translationVectors?

I placed a breakpoint and found that the internal Data property is null. See screenshot of VS debugger:

That explains why I cannot access it. But why is it null in the first place?

Answer 1

It is because of bug in EmguCV. You are calling

public static double CalibrateCamera(
   MCvPoint3D32f[][] objectPoints,
   PointF[][] imagePoints,
   Size imageSize,
   IInputOutputArray cameraMatrix,
   IInputOutputArray distortionCoeffs,
   CvEnum.CalibType calibrationType,
   MCvTermCriteria termCriteria,
   out Mat[] rotationVectors,
   out Mat[] translationVectors)

inside this method there is a call to

public static double CalibrateCamera(
   IInputArray objectPoints,
   IInputArray imagePoints,
   Size imageSize,
   IInputOutputArray cameraMatrix,
   IInputOutputArray distortionCoeffs,
   IOutputArray rotationVectors,
   IOutputArray translationVectors,
   CvEnum.CalibType flags,
   MCvTermCriteria termCriteria)

IOutputArray rotationVectors should be copied to Mat[] rotationVectors. The same thing in case of translationVectors. The problem is in this loop.

There is

for (int i = 0; i < imageCount; i++)
{
   rotationVectors[i] = new Mat();
   using (Mat matR = rotationVectors[i]) // <- bug
      matR.CopyTo(rotationVectors[i]);
   translationVectors[i] = new Mat();
   using (Mat matT = translationVectors[i]) // <- bug
      matT.CopyTo(translationVectors[i]);                
}

and there should be

for (int i = 0; i < imageCount; i++)
{
   rotationVectors[i] = new Mat();
   using (Mat matR = rVecs[i]) 
      matR.CopyTo(rotationVectors[i]);
   translationVectors[i] = new Mat();
   using (Mat matT = tVecs[i]) 
      matT.CopyTo(translationVectors[i]);                
}

Finally to get rotation and translation values you can copy data using DataPointer

var rotation = new Matrix<float>(rotationVectors[0].Rows, rotationVectors[0].Cols, rotationVectors[0].DataPointer);