CODEWITHSUNDEEP
tensorflowdeep-learningbatch-normalization
Bill
Bill

Reputation: 466

Passing beta and gamma into batch normalization

I have a very specific use case where I need to manually pass in the parameters for the affine transform for batch norm (gamma and beta). As far as I can tell, neither tf.contrib.layers.batch_norm nor tf.layers.batch_normalization allow this (I believe the accepted answer to this related question is incorrect, at least for recent versions of Tensorflow: How to give beta and gamma in tf.contrib.layers.batch_norm).

Is there any way to accomplish this without manually defining a custom batch norm op that uses tf.nn.batch_normalization (I'd like Tensorflow to take care of maintaining the moving averages for mean and variance if possible)?

Upvotes: 1

Views: 704

Answers (1)

Olivier Dehaene
Olivier Dehaene

Reputation: 1680

If you want to pass a specific value of Beta and Gamma, you can modify their initialiser like so :

tf.layers.batch_normalization(beta_initializer=tf.constant_initializer(your_beta),
                              gamma_initializer=tf.constant_initializer(your_gamma)
                              trainable=??)

Note the trainable argument. This will either allow the model to learn the optimal values of Beta and Gamma, or fix them to the value you provided.

Upvotes: 0

Related Questions