Transformation of camera calibration patterns

Question

I use camera calibration in matlab to detect some checkerboard patterns, after

figure; showExtrinsics(cameraParams, 'CameraCentric');

Now, I want to rotate the checkerboard patterns around the x-axis such that all of them have nearly the same y coordinates in the camera frame.

Method: I get the positions of all patterns in the camera's frame. Then I do optimization,where the objective function is to minimize variance in y and the variable is rotation about x ranging from o to 360.

Problem: But when I plot the transformed y-coordinates, they are even nearly in a line.

Code:

Get the checkerboad points:

%% Get rotation and translation matrices for each image; 
T_cw=cell(num_imgs,1); % stores camera to world rotation and translation for each image
pixel_coordinates=zeros(num_imgs,2); % stores the pixel coordinates of each checkerboard origin
for ii=1:num_imgs,
    % Calibrate the camera
    im=imread(list_imgs_path{ii}); 
    [imagePoints, boardSize] = detectCheckerboardPoints(im);
    [r_wc, t_wc] = extrinsics(imagePoints, worldPoints, cameraParams);
    T_wc=[r_wc,t_wc';0 0 0 1];
    % World to camera matrix
    T_cw{ii} = inv(T_wc); 
    t_cw{ii}=T_cw{ii}(1:3,4); % x,y,z coordinates in camera's frame
end

Data(num_imgs=10):

t_cw
[-1072.01388542262;1312.20387622761;-1853.34408157349]  
[-1052.07856598756;1269.03455126794;-1826.73576892251]  
[-1091.85978641218;1351.08261414473;-1668.88197803184]
[-1337.56358084648;1373.78548638383;-1396.87603554914]  
[-1555.19509876309;1261.60428874489;-1174.63047408086]
[-1592.39596647158;1066.82210015055;-1165.34417772659]  
[-1523.84307918660;963.781819272748;-1207.27444716506]  
[-1614.00792252030;893.962075837621;-1114.73528985018]
[-1781.83112607964;708.973204727939;-797.185326205240]
[-1781.83112607964;708.973204727939;-797.185326205240]

Main code (Optimization and transformation):

%% Get theta for rotation
f_obj = @(x)var_ycors(x,t_cw);
opt_theta = fminbnd(f_obj,0,360);
%% Plotting (rotate ycor and check to fix theta)
y_rotated=zeros(1,num_imgs);
for ii=1:num_imgs,
    y_rotated(ii)=rotate_cor(opt_theta,t_cw{ii});
end
plot(1:numel(y_rotated),y_rotated);


function var_computed=var_ycors(theta,t_cw)
ycor=zeros(1,numel(t_cw));
for ii =1:numel(t_cw),
    ycor(ii)=rotate_cor(theta,t_cw{ii});
end
var_computed=var(ycor);
end

function ycor=rotate_cor(theta,mat)
r_x=[1 0 0; 0 cosd(theta) -sind(theta); 0 sind(theta) cosd(theta)];
rotate_mat=mat'*r_x;
ycor=rotate_mat(2);
end

Answer 1

This is a clear eigenvector problem!

Take your centroids:

t_cw=[-1072.01388542262;1312.20387622761;-1853.34408157349  
-1052.07856598756;1269.03455126794;-1826.73576892251 
-1091.85978641218;1351.08261414473;-1668.88197803184
-1337.56358084648;1373.78548638383;-1396.87603554914  
-1555.19509876309;1261.60428874489;-1174.63047408086
-1592.39596647158;1066.82210015055;-1165.34417772659  
-1523.84307918660;963.781819272748;-1207.27444716506  
-1614.00792252030;893.962075837621;-1114.73528985018
-1781.83112607964;708.973204727939;-797.185326205240
-1781.83112607964;708.973204727939;-797.185326205240];
t_cw=reshape(t_cw,[3,10])';

compute PCA on them, so we know the principal conponents:

[R]=pca(t_cw);

And.... thats it! R is now the transformation matrix between your original points and the rotated coordinate system. As an example, I will draw in red the old points and in blue the new ones:

hold on

plot3(t_cw(:,1),t_cw(:,2),t_cw(:,3),'ro')
trans=t_cw*R;
plot3(trans(:,1),trans(:,2),trans(:,3),'bo')

You can see that now the blue ones are in a plane, with the best possible fit to the X direction. If you want them in Y direction, just rotate 90 degrees in Z (I am sure you can figure out how to do this with 2 minutes of Google ;) ).

Note: This is mathematically the best possible fit. I know they are not as "in a row" as one would like, but this is because of the data, this is honestly the best possible fit, as that is what the eigenvectors are!