CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-vertex-aiimage-masking
Ben P
Ben P

Reputation: 3379

How to correctly generate background images with Vertex Ai and masking

I am trying to take an image of pair or trainers and use generative Ai within Google Cloud to create a new background. I have been able to remove the existing background (example 2, this is a PNG but the background appears black when pasting it) and create two masks (examples 3 (standard) and 6 (inverted)), following the steps outlines on this blog post.

I am then taking the example code found on the Google documentation here: https://cloud.google.com/vertex-ai/generative-ai/docs/samples/generativeaionvertexai-imagen-edit-image-mask

To bring me to this point:

from vertexai.preview.vision_models import Image, ImageGenerationModel

project_id = "x"
input_file = "trainers_nobg_color.png"
mask_file = "trainers_mask.png"
output_file = "trainers_edited.png"
prompt = "A sandy beach background."

vertexai.init(project=project_id, location="europe-west2")

model = ImageGenerationModel.from_pretrained("imagegeneration@002")
base_img = Image.load_from_file(input_file)
mask_img = Image.load_from_file(mask_file)

images = model.edit_image(
    base_image=base_img,
    mask=mask_img,
    prompt=prompt,
    # Optional parameters
    seed=2,
    # Controls the strength of the prompt.
    # -- 0-9 (low strength), 10-20 (medium strength), 21+ (high strength)
    guidance_scale=21,
    number_of_images=1,
)

images[0].save(location=output_file, include_generation_parameters=False)
images[0].show()

It runs successfully, but I am not getting the desired image output. Examples can be seen in images 1 and 4. In image 1 it seems to be using my prompt to edit the trainers and not touching the background at all (I set the image background to green in this instance). Settings for image 1 were:

input_file = "trainers_nobg.png"
mask_file = "trainers_mask.png"
output_file = "trainers_edited.png"
prompt = "A sandy beach background."

In image 4 I am getting a weird hybrid where the trainers can be just about seen where the waves meet the beach. Settings for image 1 were:

input_file = "trainers_nobg.png"
mask_file = "trainers_mask_inv.png"
output_file = "trainers_edited.png"
prompt = "A sandy beach background."

My desired output is for the trainers to be completely untouched, while the background is generated by the prompt, is there something wrong with the way I am feeding my masks? Is there some other setting that I need to specify?

Image examples of outputs from Vertex image editing.

Upvotes: 1

Views: 340

Answers (1)

gurs
gurs

Reputation: 39

I am guessing that you already figured this out, since you posted 4 months ago, but if not...

You are not telling Imagen which edit mode to use, and there is an edit mode that places products on different backgrounds. Use:

 edit_mode="product-image"

You provide the base_image (your product) and a description of the background. No mask file required.

Details here.

Upvotes: 2

Related Questions