CODEWITHSUNDEEP
pythontensorflow
Pablo Sanchez
Pablo Sanchez

Reputation: 333

How to apply clipping to trainable variables in TensorFlow

I would like to know how to apply clipping to a trainable variable in TensorFlow.

I have a variable z that I am training

z = tf.get_variable(...)

Then I want to optimize it but I want to keep it in the range [-1,1]. Right now, I'm doing the clipping as shown below:

train_step = optimizer.minizmize(loss, var_list=[z])
z = tf.clip_by_value(z, -1, 1)

But I have the feeling that the clipping is not being performed. How should it be done?

Upvotes: 3

Views: 1959

Answers (1)

xdurch0
xdurch0

Reputation: 10474

Your attempt at clipping does not work because tf.clip_by_value just returns a new tensor that will hold the clipped value of the variable, however the variable itself will not be affected. I.e. after your code snippet the Python variable z does not point to the originally created Tensorflow variable anymore.

If you want to do this manually you should use tf.assign to actually assign the clipped value to the variable. However, the most convenient way is likely to use the constraint parameter of get_variable. Please check the docs. Something like this should work:

z = tf.get_variable(..., constraint=lambda x: tf.clip_by_value(x, -1., 1.)

This should apply the function passed to constraint after each minimize call.

Upvotes: 3

Related Questions