CODEWITHSUNDEEP
opencvstereo-3d
beco
beco

Reputation: 11

OpenCV stereoCalibrate returns high RMS error

i have some problems while computing rectification of stereo pairs with OpenCV: stereoCalibrate returns an high rms error and i obtain bad rectification pairs. I tried both my rectification program and the stereo_calib.cpp provided with opencv. They both returns similiar rms errors. Plus, i ran my program with the sample stereo pairs in opencv/sample/cpp and i got correctly rectified images. So i believe the problem is in the way i take stereo picture, is it possible?

I use the stereo camera of an Htc Evo 3D (a 3D smartphone) taking pictures of a chessboard pattern. I tried to change the number and the set of pictures used as input but the littlest stereoCalibration rms i got was around 1.5 and the rectified images are totally wrong.
Is there any "suggested" way to take a set of pictures for calibration? Thanks, Andrea

Upvotes: 1

Views: 7582

Answers (5)

Raj K
Raj K

Reputation: 325

In my case, I was extracting frames from two video feeds which I thought were synced - however, they were not. After fixing the syncing problem, my RMSE decreased from over 20 px to ~1 px.

Upvotes: 0

Phann
Phann

Reputation: 1337

Check the detected chessboard pattern pairs (e.g. with drawChessboardCorners). In some cases the order of the points differs in both images (see figure; in top image the pattern is recognised from right to left, in the bottom picture from left to right). If that is the case, then the points do not longer correspond to each other. This leads to a high average reprojection error in stereoCalibrate. When checking the images for each camera individually with calibrateCamera the rms could at the same time be very low for the intrinsics, as the differing order in the stereo images doesn't matter here.

enter image description here

I solved the problem by checking the found chessboard patterns and removing the image pairs where the pattern is detected in different order. In my case this improved the rms from 15 (only one or few wrong pairs) to less than 1. In cases where the pairs are corrupted more often or when using higher resolution images (with more pixels) the error will be much higher, I guess.

Another solution could be the usage of ArUco and ChArUco patterns where the order shouldn't be ambiguous, but I haven't tested that approach.

Upvotes: 5

YuZ
YuZ

Reputation: 445

I take 9 calibration pictures of a 12x18 corners chessboard, forbid fisheye lenses k3=0, and refine the found corners to subpixel positions. The minimum achieved error is something like 0.2 at a resolution 640x480. I suggest to look for cornerSubpix(), TermCritiria and flags for stereocalibrate() in opencv doc. Code looks like this:

namedWindow( "Webcaml", CV_WINDOW_AUTOSIZE );
namedWindow( "Webcamr", CV_WINDOW_AUTOSIZE );
int successes=0;
int n_boards=9;
while(successes<n_boards)
{
    video_l.read( frame_l );
    video_r.read( frame_r );
    if( framenumber++ % board_dt == 0 && framenumber != 0)
    {
        bool patternfoundl = findChessboardCorners( frame_l, Size( board_w, board_h ), corners_l, CV_CALIB_CB_FILTER_QUADS + CV_CALIB_CB_ADAPTIVE_THRESH + CALIB_CB_FAST_CHECK );
        bool patternfoundr = findChessboardCorners( frame_r, Size( board_w, board_h ), corners_r, CV_CALIB_CB_FILTER_QUADS + CV_CALIB_CB_ADAPTIVE_THRESH + CALIB_CB_FAST_CHECK );
        if(patternfoundl && patternfoundr)
        {
            cvtColor(frame_l,frame_l_grey,CV_RGB2GRAY);
            cvtColor(frame_r,frame_r_grey,CV_RGB2GRAY);
            cornerSubPix(frame_l_grey,corners_l,Size(6,6),Size(-1,-1),TermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER,50000000000,0.0000000000001));
            cornerSubPix(frame_r_grey,corners_r,Size(6,6),Size(-1,-1),TermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER,50000000000,0.0000000000001));
        }
        drawChessboardCorners( frame_l, Size( board_w, board_h ), corners_l, patternfoundl );
        drawChessboardCorners( frame_r, Size( board_w, board_h ), corners_r, patternfoundr );
        imshow( "Webcaml", frame_l );
        imshow( "Webcamr", frame_r );
        if( corners_l.size() == (board_w*board_h) && corners_r.size() == (board_w*board_h) )
        {
            cornervector_l.push_back( corners_l );
            cornervector_r.push_back( corners_r );
            point3dvector.push_back( point3d );
            successes++;
            int c = cvWaitKey( 1000 );
        }
    }
    else
    {
        imshow( "Webcaml", frame_l);
        imshow( "Webcamr", frame_r);
    }
    char c = cvWaitKey( 1 );
    if( c == 27 )break;
}
destroyAllWindows();
rms_l = calibrateCamera( point3dvector, cornervector_l, Size( video_r.get( CV_CAP_PROP_FRAME_WIDTH ), video_r.get( CV_CAP_PROP_FRAME_HEIGHT )),
                        intrinsics_l, distortion_l, rvecs_l, tvecs_l, CV_CALIB_FIX_K3 , cvTermCriteria( CV_TERMCRIT_ITER+CV_TERMCRIT_EPS,150000000000000000,DBL_EPSILON ) );
rms_r = calibrateCamera( point3dvector, cornervector_r, Size( video_r.get( CV_CAP_PROP_FRAME_WIDTH ), video_r.get( CV_CAP_PROP_FRAME_HEIGHT )),
                        intrinsics_r, distortion_r, rvecs_r, tvecs_r, CV_CALIB_FIX_K3 , cvTermCriteria( CV_TERMCRIT_ITER+CV_TERMCRIT_EPS,150000000000000000,DBL_EPSILON ) );
cout << "intrinsic_l       = " << endl << format(intrinsics_l,"C"     ) << endl << endl;
cout << "intrinsic_r       = " << endl << format(intrinsics_r,"C"     ) << endl << endl;
cout << "distortion_l        = " << endl << format(distortion_l,"C"     ) << endl << endl;
cout << "distortion_r        = " << endl << format(distortion_r,"C"     ) << endl << endl;
cout << "tvecs_l      = " << endl << format(tvecs_l[0],"C"     ) << endl << endl;
cout << "rvecs_l      = " << endl << format(rvecs_l[0],"C"     ) << endl << endl;
double rms_stereo = stereoCalibrate( point3dvector, cornervector_l, cornervector_r, intrinsics_l, distortion_l, intrinsics_r, distortion_r,
                                    Size( video_r.get( CV_CAP_PROP_FRAME_WIDTH ),video_r.get( CV_CAP_PROP_FRAME_HEIGHT )), R, T, E, F, 
                                    TermCriteria( TermCriteria::COUNT+TermCriteria::EPS, 150000000000000000,DBL_EPSILON ), CV_CALIB_FIX_K3+CV_CALIB_FIX_INTRINSIC);
Rodrigues(R, R);
cout << "R       = " << endl << format(R,"C"     ) << endl << endl;
cout << "T        = " << endl << format(T,"C"     ) << endl << endl;
cout << "E       = " << endl << format(E,"C"     ) << endl << endl;
cout << "F        = " << endl << format(F,"C"     ) << endl << endl;
cout << "RMS Fehler l,r,stereo: " << rms_l << rms_r << rms_stereo << endl;
stereoRectify( intrinsics_l, distortion_l, intrinsics_r, distortion_r, Size( video_r.get( CV_CAP_PROP_FRAME_WIDTH ),video_r.get( CV_CAP_PROP_FRAME_HEIGHT )), R, T,
                rectify_l, rectify_r, projection_l, projection_r, Q);

Upvotes: 1

beco
beco

Reputation: 11

Hi made some more research and found this post on OpenCV answer. The phone seems to adjust the intrinsic matrix continuosly changing focus and i am unable to make a good calibration.

Upvotes: 0

QED
QED

Reputation: 808

Check out the frequent mistakes guide here to see if you have made any of these common errors.

http://www.cvlibs.net/software/calibration/mistakes.php

Also when calibrating a stereo camera, you may want to first calibrate each camera on it's own, then calibrate them as a stereo pair given their pre-estimated camera matrices.

another alternative toolbox, as well as cvlibs (link above), is here:

http://www.vision.caltech.edu/bouguetj/calib_doc/

cheers

Upvotes: 1

Related Questions