CODEWITHSUNDEEP
tensorflowdeep-learningbatch-normalizationstyle-transfer
Vitor Guizilini
Vitor Guizilini

Reputation: 31

Tiled Instance Normalization

I am currently implementing a few image style transfer algorithms for Tensorflow, but I would like to do it in tiles, so I don't have to run the entire image through the network. Everything works fine, however each image is normalized differently, according to its own statistics, which results in tiles with slightly different characteristics.

I am certain that the only issue is instance normalization, since if I feed the true values (obtained from the entire image) to each tile calculation the result is perfect, however I still have to run the entire image through the network to calculate these values. I also tried calculating these values using a downsampled version of the image, but resolution suffers a lot.

So my question is: is it possible to estimate mean and variance values for instance normalization without feeding the entire image through the network?

Upvotes: 0

Views: 426

Answers (1)

Peter Szoldan
Peter Szoldan

Reputation: 4868

You can take a random sample of the pixels of the image, and use the sample mean and sample variance to normalize the whole image. It will not be perfect, but the larger the sample, the better. A few hundred pixels will probably suffice, maybe even less, but you need to experiment.

Use tf.random_uniform() to get random X and Y coordinates, and then use tf.gather_nd() to get the pixel values at the given coordinates.

Upvotes: 0

Related Questions