CODEWITHSUNDEEP
opencvimage-processing
David Mennenoh
David Mennenoh

Reputation: 290

OpenCV - align stack of images - different cameras

enter image description here

We have this camera array arranged in an arc around a person (red dot). Think The Matrix - each camera fires at the same time and then we create an animated gif from the output. The problem is that it is near impossible to align the cameras exactly and so I am looking for a way in OpenCV to align the images better and make it smoother.

Looking for general steps. I'm unsure of the order I would do it. If I start with image 1 and match 2 to it, then 2 is further from three than it was at the start. And so matching 3 to 2 would be more change... and the error would propagate. I have seen similar alignments done though. Any help much appreciated.

Upvotes: 3

Views: 1483

Answers (2)

thitch
thitch

Reputation: 31

If I understand the application correctly, you should be able to obtain the relative pose of each camera in your array using homographies:

https://docs.opencv.org/3.4.0/d9/dab/tutorial_homography.html

From here, the next step would be to correct for alignment issues by estimating the transform between each camera's actual position and their 'ideal' position in the array. These ideal positions could be computed relative to a single camera, or relative to the focus point of the array (which may help simplify calculation). For each image, applying this corrective transform will result in an image that 'looks like' it was taken from the 'ideal' position.

Note that you may need to estimate relative camera pose in 3-4 array 'sections', as it looks like you have a full 180deg array (e.g. estimate homographies for 4-5 cameras at a time). As long as you have some overlap between sections it should work out.

Most of my experience with this sort of thing comes from using MATLAB's stereo camera calibrator app and related functions. Their help page gives a good overview of how to get started estimating camera pose. OpenCV has similar functionality.

https://www.mathworks.com/help/vision/ug/stereo-camera-calibrator-app.html

The cited paper by Zhang gives a great description of the mathematics of pose estimation from correspondence, if you're interested.

Upvotes: 3

Nejc
Nejc

Reputation: 937

Here's a thought. How about performing a quick and very simple "calibration" of the imaging system by using a single reference point?

The best thing about this is you can try it out pretty quickly and even if results are too bad for you, they can give you some more insight into the problem. But the bad thing is it may just not be good enough because it's hard to think of anything "less advanced" than this. Here's the description:

  • Remove the object from the scene
  • Place a small object (let's call it a "dot") to position that rougly corresponds to center of mass of object you are about to record (the center of area denoted by red circle).
  • Record a single image with each camera
  • Use some simple algorithm to find the position of the dot on every image
  • Compute distances from dot positions to image centers on every image
  • Shift images by (-x, -y), where (x, y) is the above mentioned distance; after that, the dot should be located in the center of every image.

When recording an actual object, use these precomputed distances to shift all images. After you translate the images, they will be roughly aligned. But since you are shooting an object that is three-dimensional and has considerable size, I am not sure whether the alignment will be very convincing ... I wonder what results you'd get, actually.

Upvotes: 2

Related Questions