CODEWITHSUNDEEP
pythonopencvimage-processing
Adorn
Adorn

Reputation: 1421

Finding complete contour based on partial information from saliency

I am using a deep learning method to find the salient objects in the image. The output of saliency finder is pixel-wise information (between 0-1). Based on some threshold, I get an area in the image. But often times, it is only a partial object. Please see the images below:

enter image description here

Image to the left is input, on second column, first row indicates output of saliency and second row is contours.

Contours is something that I tried. But have no idea how to combine saliency and contours to get an object. In the input image, I would be looking out for a full boy.

Can you suggest any method?

Upvotes: 1

Views: 150

Answers (1)

DCS
DCS

Reputation: 3394

I think salience and "getting an object", which is called semantic- ord object segmentation in the literature, are two very distinct problems, and mixing them will probably not get you good results.

If I recall correctly, saliency is very much about predicting where in an image human attention will go first. Predictors are often based on scientific theories of attention or attention mechanisms of the brain, or on whatever definition of "saliency" seems reasonable.

This is an easier problem that object segmentation. "Object" is a very high level and very semantic notion (What is an object in your example? The whole man? His face, shirt, legs, shoes?). Semantic segmentation algorithms try to find precise boundaries, while saliency detectors just give you a kind of heat map. Here's a nice, fairly recent overview.

So what should you do?

  • If you are after the objects, get rid of saliency and try to run a state of the art object segmentation network. All methods are based on deep learning now.

  • If you want salient objects, start with the segmentation and then score the objects found according to their overlap with the saliency map.

Don't go down the rabbit hole of getting objects from contours. Won't work. People have done that for years pre deep-learning. Like object classification, it's one of the areas in computer vision where deep learning has brought a huge step forward, although I would not at all consider it solved.

Upvotes: 1

Related Questions