CODEWITHSUNDEEP
pythontensorflowkerasimage-segmentation
Nawra C
Nawra C

Reputation: 166

Inaccurate masks with Mask-RCNN: Stairs effect and sudden stops

I've been using matterport's Mask R-CNN to train on a custom dataset. However, there seem to be some parameters that i failed to correctly define because on practically all of the images, the bottom or top of the object's mask is cut off: innacurate prediction: mask is cut-off on the bottom

As you can see, the bounding box is fine since it covers the whole blade, but the mask seems to suddenly stop in a horizontal line on the bottom.

On another hand, there is a stair-like effect on masks of larger and curvier objects such as this one (in addition to the bottom and top cut-offs): enter image description here

It doesn't feel like the problem comes from the amount of training. These two predictions come from 6 epochs of 7000 steps per epoch (took around 17 hours). And the problem appears from early stage and persists along all the epochs.

Any idea on what changes to make ?

Upvotes: 2

Views: 2268

Answers (1)

Frederik Bode
Frederik Bode

Reputation: 2744

This is simply how Mask-RCNN works, and is a known side effect. Nothing you can do implementation wise to make it not appear. PointRend discusses the problem (proving that it's not just you), and also proposes their own algorithm (which is an extension to Mask-RCNN) to solve it. In the image below you can see an image from that paper. On the top left they ran Mask-RCNN on a 28x28 image, in which you can see the staircasing as well. The other images relate to how they solve it.

The bad news is of course that it's not easy to just slam the PointRend code onto Mask-RCNN, I at least don't know any great implementations. The PointRend github itself also doesn't allow for any retraining on your own data.

this

EDIT:

For postprocessing, there's a lot of approaches you can take. How about this one:

enter image description here

Upvotes: 4

Related Questions