CODEWITHSUNDEEP
tensorflow
deepskills
deepskills

Reputation: 3

Freezing specific values in a weight matrix in tensorflow

Assuming I have a a weight matrix that looks like [[a , b ], [c, d]], is it possible in Tensorflow to fix the values of b and c to zero such that they don't change during optimization?

Upvotes: 0

Views: 645

Answers (2)

Giuseppe Marra
Giuseppe Marra

Reputation: 1104

Some sample code:

A = tf.Variable([[1., 0.], [3., 0.]])
A1 = A[:,0:1] # just some slicing of your variable
A2 = A[:,1:2]
A2_stop = tf.stop_gradient(tf.identity(A2)) 
A = tf.concat((A1, A2_stop), axis=1)

Actually, tf.identity is needed to stop the gradient before A2.

Upvotes: 2

BlessedKey
BlessedKey

Reputation: 1635

There are three ways to do this, you can

  1. Break apart your weight matrix into multiple variables, and make only some of them trainable.
  2. Hack the gradient calculation to be zero for the constant elements.
  3. Hack the gradient application to reset the values of the constant elements.

Upvotes: 1

Related Questions