Camera Calibration in Natural Marker tracking

Question

Is it necessary to calibrate the camera if I were to implement a natural marker tracker?

Actually I don't quite get the idea of camera calibration although I have read that it is required for augmenting 3d/2d objects onto the image feed.

Answer 1

Camera-calibration means finding intrinsic parameters of the camera. It is necessary, of course, if you want to detect natural features sucessfully, and here I explain why.

Then you only have to look for extrinsic parameters. You only have to do calibration once, as the camera is always the same (considering you cannot zoom in/out, change focal length, etc). Without camera calibration you will have many problems for the natural features tracking task, as it is more challenging than fiducials tracking.

In the link I passed you, you will also find how to calculate the pose from a planar marker. It is theoretical, but you can find a lot of code in the web. If you need more help tell me, I can explain in more detail if necessary.

Answer 2

Is it necessary? No.

Is it useful? You bet. The rule of thumb is ALWAYS, if you can perform camera calibration for your stationary camera, do it.

You can do many things with such information: remove distortion, get distance in some type of metric space, ... Most trackers have an underlying assumption/models, these models are best fit when the data is in a space where the model makes sense. Camera calibration is one easy way to achieve this.

Answer 3

Strictly speaking, you could detect features, do pattern matching to recognize the marker and then track those features without camera calibration. Calibration allows us to determine both intrinsic (e.g. distortion coefficients) and extrinsic (e.g. rotation) camera parameters, which are required when someone is to determine marker boundaries or perform 3D pose estimation.