OpenCV GrabCut doesn't update mask when on GC_INIT_WITH_MASK mode

Question

I followed this opencv tutorial and used my own image and mask to improve segmentation using opencv grabcut algorithm, but the resulted mask is the same as the initial mask.

Code:

import cv2

init_mask = cv2.imread('/path/to/mask.png',0)
img = cv2.imread('/path/to/image.png')

mask = np.zeros(image.shape[:2],np.uint8)
mask[init_mask == 255] = 1

bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)

mask, bgdModel, fgdModel = 
cv2.grabCut(img,mask,None,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_MASK)

mask = np.where((mask==2)|(mask==0),0,1).astype('uint8')
mask[mask == 1] = 255

np.all(mask==init_mask) # True

Initial mask:

Image:

Thank you!

Answer 1

Mask for cv2.grabCut can take four values:

• 0: Ground truth background
• 1: Ground truth foreground
• 2: Probable background
• 3: Probable foreground

This is documented here.

In your case it's not doing anything because the mask is only zero and one, so you're telling the function you're certain of the current segmentation. If you set the rest of the mask image to 2, it will fit a background model to that, and the algorithm can run.

You can still set some pixels to true background, if you so desire.

import cv2

init_mask = cv2.imread('mask.png',0)
img = cv2.imread('image.png')

mask = np.zeros(img.shape[:2],np.uint8)
mask[init_mask == 255] = 1
mask[init_mask == 0] = 2 #Guess everything else is background

bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)

mask, bgdModel, fgdModel = cv2.grabCut(img,mask,None,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_MASK)

mask = np.where((mask==2)|(mask==0),0,1).astype('uint8')
mask[mask == 1] = 255