Make an image's background transparent or white programmatically

Question

If one is creating an application in which one would like to make the "background" of an image transparent or white, is there a way to do this?

For example :

In the image, http://upload.wikimedia.org/wikipedia/commons/b/b9/Bronze_Statuette_of_a_Veiled_and_Masked_Dancer_1.jpg , I would like to transform the image programmatically, such that only the statue remains, and the background (i.e. the rest of the image) is all white, or transparent.

Also, the user might point out to the part of the image that should "remain", and the rest should be all white or transparent.

How can this be done? Also, if there is a suitable java library or piece of code, it would be helpful.

Regards

Answer 1

For that particular image there are several methods that could solve part of the problem. Maybe if you combine several methods and offer the user interactive choices you could develop a good software in a reasonably short period of time. I'd use it!

You and Misha already discussed the first two items:

1. Edge detection using Canny or other means. I would suggest that you should work on a "raw" edge strength image rather than thresholding the edge strengths to generate a binarized image. Note that in this case the figure is in sharp focus, and much of the background is out of focus. Although it's not a general solution, for this particular image you could filter out edges that (a) belong to lines (using Hough or RANSAC) but that (b) have gradients that fall below a threshold steepness.
2. Flood fill. Misha already provided the link. Flood filling a "static" value shouldn't be too hard to implement (e.g. fill all neighboring pixels of value +/- N relative to the clicked pixel). Implementing a dynamic flood fill that could account for gradients due to lighting and 3D curvature into account is, well . . . ouch!
3. Mean shift clustering. This might work even as a first step to help clump together pixels of the same HSV values. Just eyeballing the image, though, the hue of the foreground figure and the hue of most of the background is similar. OpenCV has an implementation of mean shift. There's a video of the related CAMSHIFT algorithm at work: http://www.youtube.com/watch?v=iBOlbs8i7Og
4. Edge following for strong edges. If the user clicks close to an edge, you identify the closest strong edge and then use a contour-following (or "contour tracing") algorithm. Basic contour-following algorithms work on binary images; you could adapt your algorithm to try to follow a strong edge in RGB space. Tricky!
5. Since you're writing software for a user, devote some of your time to software usability rather than just trying to solve the general image processing problem.
6. Check for occlusion of background curves. Another way to determine if an item is part of the background is to determine if it is occluded (hidden) by a foreground object. If you find two line segments that are colinear and have gentle gradients (i.e. they're out of focus), then they may be two line segments rather than one because they are occluded by a foreground object.
7. If by any chance you can work with a camera rather than existing images, you can mimic a "light field" or plenoptic camera (such as the Lytro https://www.lytro.com/camera) by taking multiple successive images at different focus settings. This can help you identify figures at different depths based on changes in gradients. If the statue is relatively close to the camera, it will go out of focus as the camera focuses towards infinity.

Improving usability

1. Assuming you can segment the image into chunks that are reasonably distinct, prompt the user to click on chunks that belong to the same object of interest. Each chunk could have its own set of tweaking parameters for edge strength, acceptance range for color, etc. Having chunk-specific parameters can help make the software usable even if there are shadows, varying lighting, etc., that present problems for segmentation. Something like this can be done in GIMP and Photoshop by combining selections, but it's less usable than it could be.

2. For the chunks that have been identified, implement a "snap to edge" feature that helps the user moves the discovered edge curve onto the true edge curve. If the user grabs a chunk contour and drags it in one direction, the contour could snap to the next strong edge in that direction.

3. Offer a batch processing option. If a user has a series of photos taken under the same conditions, then the user-selected chunks for the first image could help guide the software in setting parameters for successive images. This isn't intended to solve the general segmentation problem, but might save the user a bit of time and effort for certain groups of images.

It's a fun problem. Good luck!

Answer 2

What you're asking for is programmatic segmentation of foreground and background. As this is an active research area, you're unlikely to find any ready-mode source code that works out of the box (especially in Java).

If you have the time, look up image segmentation and browse through relevant papers on Google Scholar. You will see that in the general case, it's not an easy problem for a computer to solve. In specific cases, you can try to take advantage of certain conditions. In the image you've specified, the background is really blurry, so the edges of the figure really stand out well. Edge detection with the Canny operator gives you this:

It's not perfect, but it's a start. Using the edge information, you can locate major external contours and extract the dancer figure.