CODEWITHSUNDEEP
c++opencvimage-processingimage-segmentation
RossFe
RossFe

Reputation: 71

OpenCV : Using a Trimap image

I found this dog and cat image dataset: The Oxford-IIIT Pet Dataset. Each image has a pixel level foreground-background segmentation (trimap) image.

Searching the internet, I saw that trimap is an image with three colors (one for the background, one for the foreground and one for the not-classified region), but here the image is all black.

Is it a mistake or is it correct? But above all I want to know if and how you can use it to get, given a normal image, a new image with the cat or dog on a black background.

Thanks.

Upvotes: 4

Views: 6830

Answers (2)

maverik89
maverik89

Reputation: 43

Struggled with this for a bit as well until I finally managed to figure it out.To load and display these files in Python you would need this 

import io
import os
import tensorflow as tf
import PIL
import matplotlib.pyplot as plt
import numpy as np

with tf.gfile.GFile("Abyssinian_1.png", 'rb') as fid:
    encoded_mask_png = fid.read()
encoded_png_io = io.BytesIO(encoded_mask_png)
mask = PIL.Image.open(encoded_png_io)            
plt.imshow( np.array(mask) / 2 * 255 )

Upvotes: 1

Mark Setchell
Mark Setchell

Reputation: 207688

The trimaps look black because they only contain pixels values ranging from 0-2 on a scale of 0-255, where:

  • 1 means "pet"
  • 2 means "background"
  • 3 means "border"

Look at the pixels in text form:

identify -verbose Abyssinian_1trimap.png  | more

Output

  Histogram:
     22938: (  1,  1,  1) #010101 gray(1)
    198766: (  2,  2,  2) #020202 gray(2)
     18296: (  3,  3,  3) #030303 gray(3)

If you contrast stretch the trimaps, you can see better. I am using the Abyssinian1 image here:

convert Abyssinian_1trimap.png -auto-level trimap.jpg

enter image description here

If you make all the 1 pixels in the trimap white and all the 2 pixels black and all the 3 pixels white and blend that with the actual photo using a darken blend, you will get what you want:

convert Abyssinian_1.jpg \( Abyssinian_1trimap.png -fill white -opaque "rgb(1,1,1)" -opaque "rgb(3,3,3)" -fill black -opaque "rgb(2,2,2)" \) -compose darken -composite pet.png

enter image description here

If you want the border as well as the pet, do this:

convert Abyssinian_1.jpg \( Abyssinian_1trimap.png -fill white -opaque "rgb(1,1,1)" -opaque "rgb(3,3,3)" -fill black -opaque "rgb(2,2,2)" \) -compose darken -composite pet.png

enter image description here

You can also experiment with blurring the mask to soften the edges:

convert Abyssinian_1.jpg \( Abyssinian_1trimap.png -fill white -opaque "rgb(1,1,1)" -fill black -opaque "rgb(3,3,3)" -opaque "rgb(2,2,2)" -blur 0x8  \) -compose darken -composite pet.png

enter image description here

Sorry, I did it with ImageMagick because I find that easier and it is installed on most Linux distros and available for macOS and Windows. The principles are the same for OpenCV.

Upvotes: 9

Related Questions