Finding patches of certain colors with a size smaller than x. (x =number of pixels)

Question

Segmentation

Blue Mask

In this example you can see the Segmentation and a mask showing all the places where the Segmentation has the color blue (0,0,155,255). There is some blue noise, represented as small blue streak just bewteen the green and red area and between the green and orange areas. I would like to remove the blue Segmentation if the segementation has a an area which is smaller than, lets say 50 pixels, and replace it by the color surronding the blue area, without mixing any colors. The end result should only conatin the 6 original colors.

Idealy I would like to perform this process for all 6 colors in the image.

How would I have to go about this, is there any inbuilt function that can do this?

Answer 1

Finding a suitable algorithm to fill in the small contours, colours around the object seemed to be too complicated so I came up with ( with the help of Todor) this:

import os
import numpy as np
from PIL import Image
import time
import cv2
from joblib import Parallel, delayed

root = 'Mask/'
files = os.listdir(root)

def despeckling(file):
#for file in files: -> if you don't want to use multiple threads to compute this.
    
    imgpath = os.path.join(root, file)     
    img1 = Image.open(imgpath)             #opening file 
    img1 = img1.convert("RGB")             #convert to rgb
    pixels1 = img1.load()
    
# blue 2 green
    newimgarray0 = np.asarray(img1)
    for y in range(img1.size[1]):                  #returning an binary img with...
        for x in range(img1.size[0]):
            if pixels1 [x,y] != (0, 0, 155):       #the color you want to isolate, and ...
                pixels1[x,y] = (0,0,0)             #the background color  (black)
    img1arr = np.asarray(img1)
    grayarr1 = cv2.cvtColor(img1arr, cv2.COLOR_RGB2GRAY)   # you have to convert to grayscale as cv2.find contours can't process anything else
    contours1, hierachy = cv2.findContours(grayarr1,cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE) # returning the contours with out any extrapolation (cv2.CHAIN_APPROX_NONE) , disregarding hierachy (RETR_LIST)

    shapes1 = []                          # empty array to store the Contours in
    for contour1 in contours1:
        if cv2.contourArea(contour1) < 1000:         # specifying the minimum contour area( here it is 1000)
            shapes1.append(contour1)             # storing the contours in the shapes1

    newimgarray1 = cv2.fillPoly(newimgarray0, shapes1, color=(0,174,0))     # filling the contours, which are deemed to smal (<1000) with next color inline
    newimg1 = Image.fromarray((newimgarray1))
    
#repeat for all colors until no small patch is left.
 
    newdir = 'despeckled/'
    newimg1.save(os.path.join(newdir, file))

run = Parallel(n_jobs=-1) (delayed(despeckling)(file) for file in files)   #parallisation of the process 

As I have 6 colors, this would be the order I would go through

blue -> green, green -> orange, orange -> red, red -> purple, purple -> blue, blue -> green, green -> orange, orange -> red, red -> purple

This way I can ensure, that all the small patches now belong to one bigger patch.

There are for sure better ways to do this, but this was the easiest for me since I am still a noob. :D

Answer 2

I would apply findContours on the (thresholded) masks, per each color, and collect the segmented representations. Then render each of the colors separately as you've done with the blue mask.

Then I'd use these functions https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_contours/py_contour_features/py_contour_features.html#contour-features

E.g. filtering by: area = cv2.contourArea(cnt) mark the small regions.

That is - iterate the contours and compare if area < ... --> collect:

For each selected small region, you may check the surroundings, which colors are adjacent. That could be done e.g. by sampling some point from the contour (it is a list of coordinates) and scanning in various directions and comparing the colour until finding a different one. That could be helped by finding the extreme points and starting from there, see below:

  #... produce masked image for each color, put in masks = [] ...
    #... colors = [] ... per each mask/segmented region etc.
    for m in masks:
      bw = cv2.cvtColor(m,cv2.COLOR_BGR2GRAY)
      ret, thresh = cv2.threshold(bw,127,255,0) # or whatever appropriate params 
      contours, hierarchy = cv2.findContours(thresh, 1, 2)
      streaks = []
      for c in contours:
         if cv2.contourArea(c) < minSize:
           streaks.append(c)
           # or process directly, maybe a function here, or could simplify the contour:
           # reduce the number of points: 
           # epsilon = 0.1*cv2.arcLength(cnt,True)
           # approx = cv2.approxPolyDP(cnt,epsilon,True)
           for x,y in c: # or on the approx or with skipping ... or one point may be enough
               # check with offset... Scan in some direction until black/different color or pointPolygonTest() is False etc.
       '''

The extreme points could be found which may make the scanning more efficient - c is a contour:

leftmost = tuple(c[c[:,:,].argmin()][0]
rightmost = tuple(c[c[:,:,].argmax()][0]

So having the leftmost coordinate of the contour, the scan should be to the left and for the rightmost - to the right etc. There are bordercases when the small regions are near the border of the image, then the search should iterate over the directions.

Then you can change the color of these small regions to that adjacent one - either in the representation (some class or a tuple) or directly in the image with cv2.fillPoly(...). fillPoly may be used to reconstruct the image of the segmentation.

There could be several adjacent areas with different colours, so if it matters which colour to select, it needs more specifications, e.g. comparing the areas of these adjacent ones and selecting the bigger/smaller one, random etc.