CODEWITHSUNDEEP
opencvimage-processingimage-segmentation
Bipin
Bipin

Reputation: 483

To separate objects detected from an image using PixelLib

Consider an image of a bottle. I intend to separate or extract the pixels of the bottle from the image. The bottle is recognized by image segmentation using PixelLib. The code goes as follows:

import pixellib
from pixellib.instance import instance_segmentation
import cv2

instance_seg = instance_segmentation()
instance_seg.load_model("mask_rcnn_coco.h5")
segmask, output = instance_seg.segmentImage("sample2.jpg", show_bboxes= True)
cv2.imwrite("img.jpg", output)

The 'instance_segmentation' is the class for performing instance segmentation and is imported and an instance of the class is created. 'mask_rcnn_coco.h5' is the mask r-cnn model to perform instance segmentation. 'show_bboxes = True' shows segmentation masks with bounding boxes.

I read the official documentation and there is no mention of how to separate an object detected by PixelLib.

The code is referred from link

Upvotes: 1

Views: 2523

Answers (5)

Umar Farooq Awan
Umar Farooq Awan

Reputation: 171

You can calculate the mask enter image description here like below using this answer on github, then remove the background using the following statement. bg_removed = cv2.bitwise_and(inp_img, mask)

enter image description here

Upvotes: 0

Ayoola Olafenwa
Ayoola Olafenwa

Reputation: 51

@M. Str It extracted all the objects in the image, but there is a bug somewhere that fails to return all the objects extracted. I have fixed it and it can now return all the objects extracted from the image. Upgrade to the latest version of PixelLib using;

pip3 install pixellib --upgrade

sample image

import pixellib
from pixellib.instance import instance_segmentation

segment_image = instance_segmentation()
segment_image.load_model("mask_rcnn_coco.h5")
seg, output = segment_image.segmentImage("sample4.jpg", extract_segmented_objects=True, save_extracted_objects =True,
                                         show_bboxes=True, output_image_name= "a.jpg")
res = seg["extracted_objects"]
for a in res:
    print(a.shape)

The printed shapes of the extracted objects in the image.

(238, 92, 3)
(174, 260, 3)
(117, 235, 3)

Upvotes: 0

M. Str
M. Str

Reputation: 49

In case there is more than one extracted object, the following code

segmask, output = seg.segmentImage("sample_image.jpg", extract_segmented_objects = True, save_segmented_objects = True, show_bboxes=True, output_image_name="output.jpg")

extracts only the LAST segmented object into segmask['extracted_objects'], i.e. segmask['extracted_objects'] will always contain only ONE object, no matter how many objects the algorithm identified correctly. I'm not sure if this is intended behaviour, but I wouldn't think so.

It does work, however, if this syntax is used instead:

seg.segmentImage("sample_image.jpg", extract_segmented_objects = True, save_segmented_objects = True, show_bboxes=True, output_image_name="output.jpg")

In this case the segmented objects are directly written to disk as JPG files and would need to be reloaded to be available for further processing.

Upvotes: 0

Ayoola Olafenwa
Ayoola Olafenwa

Reputation: 51

I have upgraded PixelLib to make it very convenient to extract segmented objects. Upgrade to the latest version using.

pip3 install pixellib --upgrade

In this new version of PixelLib I added the feature that makes it possible to filter unused detections and target the segmentation of a specific class.

Sample Image sample.

In the image above we want to extract the bottle object located on the table. It is possible using this code:

 import pixellib
 from pixellib.instance import instance_segmentation
    
 seg = instance_segmentation()
 seg.load_model("mask_rcnn_coco.h5")
 target_classes = seg.select_target_classes(bottle = True)
 seg.segmentImage("sample_image.jpg", segment_target_classes= target_classes, extract_segmented_objects = True, save_segmented_objects = True, show_bboxes=True,output_image_name="output.jpg")

The is very similar to the original code for performing instance segmentation of objects, except the following differences:

  • Added the function to choose a target class to segment which is bottle.
  • We added new parameters to the segmentImage function to segment the target class, extract the segmented object and save it as a separate image.

Output Image enter image description here

Extracted Segmented Object

enter image description here

You would do this to access individual values returned by the segmentImage function.

segmask, output = seg.segmentImage("sample_image.jpg", segment_target_classes= target_classes, extract_segmented_objects = True, save_segmented_objects = True, show_bboxes=True,output_image_name="output.jpg")

Access mask's values using segmask['masks'], bounding box coordinates using segmask['rois'], class ids using segmask['class_ids'] and the extracted segmented object using segmask['extracted_objects']

Upvotes: 1

Ayoola Olafenwa
Ayoola Olafenwa

Reputation: 51

This is how you can make use of PixelLib to extract a detailed information about each object in an image. Using this code

segmask, output = segment_image.segmentImage("path_to_image", show_bboxes = True)

You will obtain the following details:

  1. Each object's segmentation mask
  2. Each object's bounding box coordinate
  3. Each object's corresponding class id

You can then make use of these results in your application.

I am the creator of PixelLib.

Upvotes: 3

Related Questions