CODEWITHSUNDEEP
machine-learningimage-processingjupyter-notebooknvidiagenerative-adversarial-network
JustAMartin
JustAMartin

Reputation: 13723

How to conveniently control features in Nvidia StyleGAN?

In Nvidia's StyleGAN video presentation they show a variety of UI sliders (most probably, just for demo purposes and not because they actually had the exact same controls when developing StyleGAN) to control mixing of features:

https://www.youtube.com/watch?v=kSLJriaOumA

However, the GitHub repsoitory here https://github.com/NVlabs/stylegan seems missing any simple way to control the features.

Is there any Jupyter (or Kaggle) notebook that has UI controls for the feature control through sliders and uploaded source photos or some simple solution how to add the controls myself? How do I actually use the StyleGAN project to do more than just generate a random image?

Upvotes: 1

Views: 1083

Answers (2)

Eren Akbulut
Eren Akbulut

Reputation: 1

Actually, there are a couple of ways of controlling such StyleGAN features. See chapters 4, 6, and 9 in this survey paper https://arxiv.org/abs/2212.09102

From Chapter 6: "The main idea about editing of an image using StyleGAN is that editing is achieved through some manipulation of its latent code, thereby moving the point that represents this latent code within one of the latent spaces of StyleGAN"

Upvotes: 0

JustAMartin
JustAMartin

Reputation: 13723

In case someone else looks for the same thing.

Finally I managed to find something very close to what I was looking for - a project that allows to control StyleGAN facial features through UI:

https://github.com/SummitKwan/transparent_latent_gan

It is using CelebA dataset, so the variety is not large enough, and the features often are entangled too much. I hoped to find something similar to this solution (or as in Nvlabs's demo image mixing) for the larger Flickr-Faces-HQ dataset, but there seems to be none yet.

I guess, I'll have to study machine learning and Python myself to the level of understanding enough for adjusting transparent_latent_gan example on the Faces-HQ dataset.

It's just strange that (seemingly) nobody has yet created such a convenient UI that was featured in the Nvlabs video demo.

Upvotes: 4

Related Questions