CODEWITHSUNDEEP
pythontensorflowembedding-lookup
WeiWang
WeiWang

Reputation: 21

calculation gradients of tf.nn.embedding_lookup

import tensorflow as tf
types_lookup_table = tf.get_variable("types_lookup_table", shape=[234, 10],initializer=tf.random_normal_initializer(0, 1), dtype=tf.float32,
trainable=True)
embedding_types = tf.nn.embedding_lookup(types_lookup_table,[[2,3,4],[1,2,3]])
opt = tf.train.GradientDescentOptimizer(0.1)
gradients = tf.gradients(embedding_types, xs=types_lookup_table) 
train = opt.apply_gradients([(gradients[0], types_lookup_table)])
with tf.Session() as sess:
    tf.global_variables_initializer().run()
    h = sess.run(gradients)
    print(sess.run(train)) #right
    print(sess.run(opt.apply_gradients([(h[0],types_lookup_table)]))). # wrong

I tried to calculate the gradients of tf.nn.embedding_lookup, but the result shown is an IndexedSliceValue with 3 elements.

h

However the corresponding gradient(without sess.run) is an indexSliceValue with 1 elements.I don't know why. gradients

And therefore I can't

sess.run(opt.apply_gradients([(h[0],types_lookup_table)]) because the shape of calculation value doesn't match the shape of _types_lookup_table_, however, when I didn't calculate the intermediate value, and directly

sess.run(train) (ps:train = opt.apply_gradients([(gradients, types_lookup_table)]))

There is no problem.

But I need to calculate the intermediate value and do an add. I don't know how. Thanks

Upvotes: 2

Views: 495

Answers (0)

Related Questions