CODEWITHSUNDEEP
generategenerative-adversarial-network
Student
Student

Reputation: 27

StyleGAN how to generate B image using A source image

I am studying the StyleGAN. It is new for me, and I could not understand mix style of generating images.

In this image showed, that using A created B images. How can I do that, If I want to use A source image not from training data? image

Upvotes: 2

Views: 1838

Answers (2)

And Rew
And Rew

Reputation: 31

If you want to use a source image not from training data, you need first to invert your image into one of the latent spaces of StyleGAN (Z-, W- or S-space). Inversion methods can typically be divided into three major groups: gradient-based optimisation of the latent code, direct encoder-based mapping onto the latent code, and fine-tuning of weights of the StyleGAN generator.

It is possible to create a non-existing facial image C that shares features of both source images A and B. Such Style-Mixing editing technique is possible by using latent representations of images A and B at different layers of the StyleGAN generator.

If a code is injected into early layers of the StyleGAN generator it affects rough facial features (e.g., face posture) while injection into later layers correspond to finer details (e.g., color of the skin).

Check Figure 3 in https://arxiv.org/abs/2212.09102 for an explanatory picture and Chapter 5 "GAN Inversion to Latent Spaces of StyleGAN" for details.

Upvotes: 2

Kristoffer Sjöö
Kristoffer Sjöö

Reputation: 304

In that example image, the A source images are not training data. They are generated images, of people who do not exist. The trained network (which is just the generator part) does not take any images as input, it only takes a random 512-dimensional vector (latent).

Thus, it is impossible to do what you ask using just the StyleGAN. You would need some way to reduce an input image to a latent vector, which is hard to do and isn't guaranteed to give reasonable results anyway.

The followup paper, StyleGAN2 (https://github.com/NVlabs/stylegan2) has an architecture where it is slightly easier to try to find a matching latent for an input image, and they even talk a bit about how to do it.

Upvotes: 4

Related Questions