CODEWITHSUNDEEP
c++opencvimage-processingcomputer-vision
FvD
FvD

Reputation: 1375

Black columns in the disparity map when using StereoBM

I am trying to use StereoBM in OpenCV to extract a disparity map from a pair of images. Ignoring the bad quality of the disparity map below, you can see that it has a number of black columns on the left which correspond to the parameter ndisparities. I thought that ndisparities only tells StereoBM how "far" away it can search for a correspondence. What could cause this behaviour? It seems to limit the width of the resulting depth map, but I don't see why.

Disparity map

You can see the stereo pair here and my code below. Thanks in advance for any pointers!

Mat Limg = imread("left.jpg", CV_LOAD_IMAGE_GRAYSCALE);
Mat Rimg = imread("right.jpg", CV_LOAD_IMAGE_GRAYSCALE);

Mat disp(Limg.size(), CV_16SC1), disp8U;
int ndisparities = 512;
StereoBM SBM(StereoBM::BASIC_PRESET, ndisparities , 11);
SBM(Limg, Rimg, disp, CV_16S);

double minVal, maxVal;
minMaxLoc( disp, &minVal, &maxVal );
disp.convertTo( disp8U, CV_8UC1, 255/(maxVal - minVal));
imshow("disparity", disp8U)

Upvotes: 3

Views: 5266

Answers (2)

Mandelmus100
Mandelmus100

Reputation: 105

I would consider this behaviour a bug (or at least an overly restrictive design choice) by OpenCV.

Here is the default behaviour you're describing, where the estimated disparity map returned by cv2.StereoSGBM_create and stereo.compute is missing the left-most numDisparities columns. I'm using a synthetic scene here for which I have a perfect ground truth disparity map for comparison. The third image shows the absolute difference or error between the ground truth disparity and the estimated disparity.

OpenCV's default disparity map behaviour

Now here is what I think it should look like. Of course the parts in the bottom left corner cannot be matched successfully because they're not visible in both images, but the parts higher up in many of those left-most columns can absolutely be matched, and should be matched.

enter image description here

To achieve this, I had to employ the hack of extending the images to the left by (what in your C++ code is referred to as) ndisparities columns, then afterwards cropped those out again of the resulting disparity map.

rectified_left_gray  = np.hstack([np.zeros((height, num_disp), dtype=np.uint8), rectified_left_gray])
rectified_right_gray = np.hstack([np.zeros((height, num_disp), dtype=np.uint8), rectified_right_gray])

disparity = stereo.compute(rectified_left_gray, rectified_right_gray) / 16.0
disparity = disparity[:, num_disp:]

Upvotes: 0

Acorbe
Acorbe

Reputation: 8391

The disparity map quantifies the longitudinal (epipolar) offset between corresponding pairs of points in the left and right image.

In principle, 

 disparity := x_l - x_r

where x_l and x_r are the projections on the focal plane of the two cameras of a given point in the space. (Keep in mind that large disparities characterize pixels which are closer to the cameras. [A])

The parameter ndisparities quantifies the expected maximum disparity you can have (assuming you can neglect the min disparity).

Since you are assuming that ndisparities is your largest disparity, it holds true

    x_l - x_r < ndisparities,

i.e.

    x_r > x_l - ndisparities

therefore, it makes no sense looking for correspondences with the right image of any pixel in the first ndisparities columns of the left image: simply you can't have.

In a sense, the view cone of the right camera starts exactly ndisparities columns on the right of view cone of the left camera.

A FIX:

If you want less black columns on the left hand side of the disparity map, you need to be able to assume lower values of ndisparities.

Since ndisparities depends on the distance from the cameras of the closest object (from [A]), either put the cameras further away from the object or put the cameras closer one another.

In your very specific case you have a huge disparity (a lot of work for the stereoBM)!! The 'x' symbol in the foreground shows a disparity comparable with the scale of the image!! I believe you need to put your cameras further away.

Upvotes: 10

Related Questions