How to detect and eliminate the shadows when having the images of foreground and background?

Question

I have images of foreground and background.While i want to subtract the background from foreground,the foreground image also contains the shadow of a workpiece that i want to eliminate.

I have come across this code that does these following job.

The code for the aforementioned operation is:

`a=imread('foreground.jpg');
b=imread('background.jpg');
da=double(a);
db=double(b);
D=imabsdiff(a,b);
r=zeros(240,320);
h=a;
for ix=1:240
    for iy=1:320
        if D(ix,iy)>20
            if da(ix,iy,1)~=0&da(ix,iy,2)~=0&da(ix,iy,3)~=0
                if (db(ix,iy,1)/da(ix,iy,1)<4)&(db(ix,iy,1)/da(ix,iy,1)>1.5)
                    if (db(ix,iy,2)/da(ix,iy,2)<2.8)&(db(ix,iy,2)/da(ix,iy,2)>1.3)
                        if (db(ix,iy,3)/da(ix,iy,3)<2.05)&(db(ix,iy,3)/da(ix,iy,3)>1.14)
                            if (db(ix,iy,3)/da(ix,iy,3)<db(ix,iy,1)/da(ix,iy,1))&(db(ix,iy,3)/da(ix,iy,3)<db(ix,iy,2)/da(ix,iy,2))&(db(ix,iy,2)/da(ix,iy,2)<db(ix,iy,1)/da(ix,iy,1))
                                if abs(da(ix,iy,1)/(da(ix,iy,1)+da(ix,iy,2)+da(ix,iy,3))-db(ix,iy,1)/(db(ix,iy,1)+db(ix,iy,2)+db(ix,iy,3)))<0.129
                                    if abs(da(ix,iy,2)/(da(ix,iy,1)+da(ix,iy,2)+da(ix,iy,3))-db(ix,iy,2)/(db(ix,iy,1)+db(ix,iy,2)+db(ix,iy,3)))<0.028
                                        if abs(da(ix,iy,3)/(da(ix,iy,1)+da(ix,iy,2)+da(ix,iy,3))-db(ix,iy,3)/(db(ix,iy,1)+db(ix,iy,2)+db(ix,iy,3)))<0.143
                                            r(ix,iy)=0;
                                            h(ix,iy,1)=255;
                                            h(ix,iy,2)=255;
                                            h(ix,iy,3)=255;
                                        end
                                    end
                                end
                            end
                        end
                    end
                end                    
            end
        end
    end
end
imshow(h);
im=h-a;
imshow(im);`

As i wanted to identify the shadow in my foreground image, i applied the same code for the foreground and background images.

I have used the following code but not able to identify the shadows in my case. I am not able to obtain the shadows as you see here Could anybody suggests how could i detect and eliminate the shadows from the foreground image?

My code:

a=imread('C:\Users\PDCA 1\Desktop\Input images\case1\materialleft_mat1.jpg');
b=imread('C:\Users\PDCA 1\Desktop\Input images\case1\backgroundleft_mat1.jpg');
da=double(a);
db=double(b);
D=imabsdiff(a,b);             
N=size(D,1);
M=size(D,2);
r=zeros(N,M);
h=a;

for ix=1:N

    for iy=1:M

        if D(ix,iy)>20

              if da(ix,iy,1)~=0&da(ix,iy,2)~=0&da(ix,iy,3)~=0
                if (db(ix,iy,1)/da(ix,iy,1)<4)&(db(ix,iy,1)/da(ix,iy,1)>1.5)
                    if (db(ix,iy,2)/da(ix,iy,2)<2.8)&(db(ix,iy,2)/da(ix,iy,2)>1.3)
                        if (db(ix,iy,3)/da(ix,iy,3)<2.05)&(db(ix,iy,3)/da(ix,iy,3)>1.14)
                            if (db(ix,iy,3)/da(ix,iy,3)<db(ix,iy,1)/da(ix,iy,1))&(db(ix,iy,3)/da(ix,iy,3)<db(ix,iy,2)/da(ix,iy,2))&(db(ix,iy,2)/da(ix,iy,2)<db(ix,iy,1)/da(ix,iy,1))
                                if abs(da(ix,iy,1)/(da(ix,iy,1)+da(ix,iy,2)+da(ix,iy,3))-db(ix,iy,1)/(db(ix,iy,1)+db(ix,iy,2)+db(ix,iy,3)))<0.129
                                    if abs(da(ix,iy,2)/(da(ix,iy,1)+da(ix,iy,2)+da(ix,iy,3))-db(ix,iy,2)/(db(ix,iy,1)+db(ix,iy,2)+db(ix,iy,3)))<0.028
                                        if abs(da(ix,iy,3)/(da(ix,iy,1)+da(ix,iy,2)+da(ix,iy,3))-db(ix,iy,3)/(db(ix,iy,1)+db(ix,iy,2)+db(ix,iy,3)))<0.143
                                            r(ix,iy)=0;
                                            h(ix,iy,1)=255;
                                            h(ix,iy,2)=255;
                                            h(ix,iy,3)=255;
                                        end
                                    end
                                end
                            end
                        end
                    end
                end                    
            end
        end
    end
end
imshow(h);
im=h-a;
imshow(im);`

The results are totally wrong.The shadows identified for my image are not at all correct.

And the erroneous shadow is given as

Can anybody please tell me where i have made the mistake and how to identify the shadow for my set of images?

Edit: i also tried to convert both the images to HSV and subtract the hue component of both images to get rid of the shadows but unfortunately it leads to more stronger noises in background.

Here's the code:

fg = im2double(imread('foregroundimage.jpg'));

bg=im2double(imread(backgroundimage.jpg));

fg=rgb2hsv(fg);
bg=rgb2hsv(bg); 
f1=fg(:,:,1);

f2=bg(:,:,1);

workpiece=imabsdiff(f1,f2);

imshow(workpiece);

This is the output i acquire from the HSV conversion and subseqent subtraction of hue compontent .(As you can see,)the background noises are not fully eliminated. So anybody help me eliminate the background noise to only obtain the workpiece. Any help is much appreciated.Thanks in advance.

Thanks again.

Answer 1

The 1st 'if' selects pixels who are significantly different from the background. The 2nd 'if' in the code is probably for technical matters - a pixel of (0,0,0) is often due to a defect in the camera or imaging process. The 3nd, 4th &5th 'if's search for pixels whose red, green , and blue have a certain proportions from the corresponding background values. For example

db(ix,iy,1)/da(ix,iy,1)<4)&(db(ix,iy,1)/da(ix,iy,1)>1.5

means the red value of the pixel is between 1.5 to 4 times smaller than the red channel of the corresponding foreground pixel. Green ratio 2.8-1.3, Blue ratio 1.14-2.05.

The reasons that such ratios exist in the image is that the color of a pixel can be estimated by multiplying the reflection of the object in a certain channel, multiplied by the strength of the light source in that channel. This assumption is not always accurate as the spectrum is actually contiguous and not just RGB, but it's good enough for many purposes. It's easy to see that if you take this assumption, the ratio of shadow pixels to the background pixel should be constant for each channel.

The values of those 3 ifs are appropriate for specific light condition, in particular to bright day conditions, as the lower multiplier for the blue channel indicate a blue sky. Your image, on the other hand, seems to be taken in artificial light. Under these conditions, the multipliers should probably be much smaller, and uniform across channel.

As for the other conditions, It's hard to tell what was the poet's purpose. My guess they played with the r/g/b proportion to get good results for their image.

My suggestion to you is to start with something similar to conditions 1, 3-5, and find the right constants for them. I'd start with the same limits to all channels, as it seems that the lighting is indoor and uniform. But don't write Matlab code like that! Matlab code should use matrices, not scalars! for example

 shadow = im < bg-20;
 shadow = shadow & im(:,:,1)*4 < bg(:,:,1) & im(:,:,1)*1.5 > bg(:,:,1) ;

etc.