Calculate and plot segmentation mask pixels

Question

I have the following image:

Below is a segmentation mask within this image:

From the image above, I attempted to calculate the non-zero pixel coordinates. That is, I tried to get all of the pixels of the actual clouds that are in the mask above. When I plot these non-zero pixels, the results are this:

My question is: why are the plotted pixels in the image above not the same as from the segmentation mask, and how do I fix this? I want to get the pixels of the clouds from the segmentation mask.

Code:

# create matrix
canvas = np.zeros((img_h, img_w))

# read in image, convert to grayscale, and create mask
im_color = cv2.imread(first_row.path.values[0], cv2.COLOR_BGR2RGB)
im_gray = cv2.cvtColor(im_color, cv2.COLOR_BGR2GRAY)
_, mask = cv2.threshold(im_gray, thresh=110, maxval=255, type=cv2.THRESH_BINARY)

# bounding box coordinates for segmentation mask
ymin, ymax, xmin, xmax = 2, 222, 42, 521

# assign mask to coordinates on canvas
canvas[ymin:ymax,xmin:xmax] = mask[ymin:ymax,xmin:xmax]
y_coords, x_coords = np.where(canvas == 255.0)

# get pixel coordinates of nonzero pixels
pts = np.array(list(zip(x_coords, y_coords)))

# plot original image 
fig1 = plt.figure()
ax1 = fig1.add_axes([0,0,1,1])
ax1.set_title('Original image')
plt.imshow(im_color)

# plot segmentation mask on canvas
fig2 = plt.figure()
ax2 = fig2.add_axes([0,0,1,1])
ax2.set_title('Segmentation mask on canvas')
plt.imshow(canvas)

# plot segmentation mask pixels
fig3 = plt.figure()
ax3 = fig3.add_axes([0,0,1,1])
ax3.set_title('Segmentation mask pixel coordinates')
plt.imshow(im_color, interpolation='none')
poly = patches.Polygon(pts)
ax3.add_patch(poly)
plt.show()

Answer 1

To add upon @nathancy's answer and explicitly answer the question: the pixels calculated from the segmentation mask are indeed accurate. The reason for the jagged shape in the last plot above is because every coordinate from the mask is plotted as a single (closed) polygon, and all of these points are connected. That is the definition of a polygon:

• "A number of coplanar line segments, each connected end to end to form a closed shape." source

So the final plot represents a closed polygon containing every pixel/coordinate from the original segmentation mask.

To produce an accurate polygonal shape, you would need to find the vertices of (any possible) contours from the mask. Example:

# set up canvas for mask
mask = np.zeros((img_h, img_w), dtype=np.uint8)

# assign specific region to mask
mask[ymin:ymax,xmin:xmax] = 1

# read in image
im_color = cv2.imread(path, cv2.IMREAD_COLOR)

# convert image to grayscale
im_gray = cv2.cvtColor(im_color, cv2.COLOR_BGR2GRAY)

# create threshold of image
_, thresh = cv2.threshold(im_gray, thresh=100, maxval=255, type=cv2.THRESH_BINARY)

# create segmentation mask of thresholded image
masked_gray = cv2.bitwise_and(thresh, thresh, mask = mask)

# calculate contours of segmentation mask
contours, hierarchy = cv2.findContours(masked_gray, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

# get largest contour
tc = max(contours, key=len)

# plot contours
fig = plt.figure()
ax = fig.add_axes([0,0,1,1])
ax.set_title('Segmentation mask')
plt.imshow(masked_gray, interpolation='none')
poly = patches.Polygon(tc[:,0])
ax.add_patch(poly)
plt.show()

Result:

Answer 2

You may be looking for matplotlib.pyplot.scatter()

Seems to match the thresholded image

import cv2
from matplotlib import pyplot as plt
import numpy as np

image = cv2.imread('1.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 110, 255, cv2.THRESH_BINARY)[1]

x, y = np.where(thresh > 0)
plt.scatter(y[::-1], x)
plt.title('Plot segmentation mask pixels')
plt.show()