Compositing images by blurred mask in Numpy

Question

I have two images and a mask, all of same dimensions, as Numpy arrays:

Desired output

I would like to merge them in such a way that the output will be like this:

Current code

def merge(lena, rocket, mask):
    '''Mask init and cropping'''
    mask = np.zeros(lena.shape[:2], dtype='uint8')
    cv2.fillConvexPoly(mask, circle, 255) # might be polygon
    '''Bitwise operations'''
    lena = cv2.bitwise_or(lena, lena, mask=mask)
    mask_inv = cv2.bitwise_not(mask) # mask inverting
    rocket = cv2.bitwise_or(rocket, rocket, mask=mask_inv)
    output = cv2.bitwise_or(rocket, lena)

    return output

Current result

This code gives me this result:

Applying cv2.GaussianBlur(mask, (51,51), 0) distorts colors of overlayed image in different ways.
Other SO questions relate to similar problems but not solving exactly this type of blurred compositing.

Update: this gives same result as a current one

mask = np.zeros(lena.shape[:2], dtype='uint8')
mask = cv2.GaussianBlur(mask, (51,51), 0)
mask = mask[..., np.newaxis]
cv2.fillConvexPoly(mask, circle, 1)
output = mask * lena + (1 - mask) * rocket

Temporal solution

Possibly this is not optimal due to many conversions, please advise

mask = np.zeros(generated.shape[:2])
polygon = np.array(polygon, np.int32) # 2d array of x,y coords
cv2.fillConvexPoly(mask, polygon, 1)
mask = cv2.GaussianBlur(mask, (51, 51), 0)
mask = mask.astype('float32')
mask = cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)
foreground = cv2.multiply(lena, mask, dtype=cv2.CV_8U)
background = cv2.multiply(rocket, (1 - mask), dtype=cv2.CV_8U)
output = cv2.add(foreground, background)

Please advise how can I blur a mask, properly merge it with foreground and then overlay on background image?

Answer 1

Here is how to do that in Python/OpenCV. Your second method is close.

• Read the 3 input images
• Apply linear (or Gaussian) blur to the circle
• Stretch the circle to full dynamic range (0 to 255) as a mask
• Convert the mask to float in the range 0 to 1 as 3 channels
• Apply mask to image1 and inverted mask to image2 via multiplication and add the products together
• Convert the result to 8-bit range (0 to 255) clipping to be sure no overflow or wrap around
• Save the results

Input images:

import cv2
import numpy as np

# Read images
image1 = cv2.imread('lena_wide.jpg')
image2 = cv2.imread('rocket.jpg')
circle = cv2.imread('white_circle.jpg', cv2.IMREAD_GRAYSCALE)

# linear blur mask
mask = cv2.blur(circle, (30,30))
# alternate using Gaussian blur
#mask = cv2.GaussianBlur(circle, (0,0), sigmaX=10, sigmaY=10)

# stretch mask to full dynamic range
mask = cv2.normalize(mask, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U)

# convert mask to float in range 0 to 1
maskf = (mask/255).astype(np.float64)
maskf = cv2.merge([maskf,maskf,maskf])

# apply mask to image1 and inverted  mask to image2
result = maskf*image1 + (1-maskf)*image2
result = result.clip(0,255).astype(np.uint8)


# save results
cv2.imwrite('white_circle_ramped.jpg', mask)
cv2.imwrite('lena_wide_rocked_composited.png', result)

# show results
cv2.imshow("mask", mask)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Ramped mask image:

Result:

ADDITION:

Here is an alternate approach using mostly Numpy.

import cv2
import numpy as np

# Read images
image1 = cv2.imread('lena_wide.jpg')
image2 = cv2.imread('rocket.jpg')
circle = cv2.imread('white_circle2.jpg', cv2.IMREAD_GRAYSCALE)

# linear blur mask
mask = cv2.blur(circle, (30,30))
# alternate using Gaussian blur
#mask = cv2.GaussianBlur(circle, (0,0), sigmaX=10, sigmaY=10)

# stretch mask to full dynamic range
mask = cv2.normalize(mask, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U)

# convert mask to 3 channels
mask = cv2.merge([mask,mask,mask])

# apply mask to image1 and inverted  mask to image2
image1_masked = np.multiply(image1, mask/255).clip(0,255).astype(np.uint8)
image2_masked = np.multiply(image2, 1-mask/255).clip(0,255).astype(np.uint8)

# add the two masked images together
result = np.add(image1_masked, image2_masked)

# save results
cv2.imwrite('white_circle_ramped2.jpg', mask)
cv2.imwrite('lena_wide_rocked_composited2.png', result)

# show results
cv2.imshow("mask", mask)
cv2.imshow("image1_masked", image1_masked)
cv2.imshow("image2_masked", image2_masked)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Answer 2

You need to renormalize the mask before blending:

def blend_merge(lena, rocket, mask):
    mask = cv2.GaussianBlur(mask, (51, 51), 0)
    mask = cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)
    mask = mask.astype('float32') / 255
    foreground = cv2.multiply(lena, mask, dtype=cv2.CV_8U)
    background = cv2.multiply(rocket, (1 - mask), dtype=cv2.CV_8U)
    output = cv2.add(foreground, background)
    return output

A full working example is here.