CODEWITHSUNDEEP
opencv
Huhwha
Huhwha

Reputation: 575

Mapping from projection back to display OpenCV

I have a system where I project a set of images and then use a camera to read back what is projected. When someone is standing in front of the projection, I need to know which part of the projection they are standing in front of. I have no problem detecting the person, I just need to figure out a way to map points that are detected in the camera back to the image that is being sent to the projector.

I tried using findHomograhy (using the corners of the display image and the projected image) but the resulting matrix doesn't give accurate results. Any other ideas (or tricks to getting findHomography to work)?

Upvotes: 1

Views: 1724

Answers (1)

Sogartar
Sogartar

Reputation: 2175

I guess the image that is captured from the camera is quite blurred, noisy and distorted and I guess somehow you detect/track the corners of the projected image.

One important thing to do is to calibrate your camera to remove radial lens distortion, if you haven't done so: http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html

The projector probably does have also radial distortion, which may be possible to calibrate also with your newly calibrated camera. I don't know if you would have any luck to find ready out of the box solution though. You may have to get involved in some serious optimization problem domains similar to the camera calibration one.

After all of the above is settled and you still have problems ..., because your captured image is blurry and noisy you get errors in the corner tracking. This means that with only 4 points those errors are heavily manifested in the resulting perspective warp. The error of the extraction of point correspondences has some probability distribution. If you assume that it is normal, which in most cases it is, except the outliers, then it simply means that your standard deviation of the error is too big. Now, the law of large numbers guarantees you that if you increase the number of samples you will get close to the mean, in this case the correct projection.

What I am trying to say is, that you need more point correspondences to fight noise or more precise detection of the few. If you always/usually have some good features in the projected images and you are willing to spent computing power, you may be able to use feature matching: http://docs.opencv.org/doc/tutorials/features2d/feature_description/feature_description.html

Upvotes: 1

Related Questions