CODEWITHSUNDEEP
tensorflow
user3490622
user3490622

Reputation: 1011

Tensorflow: iterating over a Tensor for embedding lookup?

Suppose I have a matrix of N users, and each user is associated with a vector of words (translated to integers). So for example for N = 2 I'd have:

user 0 corresponds to words['20','56']

user 1 corresponds to words ['58','10','105']

So I have a list 

user_words = [['20','56'],['58','10','105']]

Suppose further I created a 100-column embedding matrix (word_emb) for these words. I'd like to look up the (mean) embeddings of each of the user vectors and create a new Tensor, whose shape I would expect to be [2,100]. I tried doing this:

word_vec = []
for word_sequence_i in tf.map_fn(lambda x: x, user_words):
    all_word_vecs = tf.nn.embedding_lookup(word_emb, word_sequence_i)
    word_vec.append( tf.reduce_mean(all_word_vecs, 1))

But this gives me an error:

TypeError: `Tensor` objects are not iterable when eager execution is not enabled. To iterate over this tensor use `tf.map_fn`.

I thought I already was using tf.map_fn above! So what is Tensorflow complaining about? Is there even a way to do what I am trying to do?

Thanks so much!

Upvotes: 0

Views: 1038

Answers (1)

ash
ash

Reputation: 6751

tf.map_fn returns a Tensor object itself, which is a symbolic reference to a value that will be computed at Session.run() time. You can see this with type(tf.map_fn(lambda x: x, user_words)). So, it's the iteration implied in for word_sequence_i in tf.map_fn(...) that is generating the error.

Perhaps what you're looking for is something like:

all_word_vecs = tf.map_fn(lambda x: tf.nn.embedding_lookup(word_emb, x), user_words)
word_vec = tf.reduce_mean(all_word_vecs, axis=1)

On a related note, if this distinction between graph construction and execution is getting bothersome, you might want to give TensorFlow's eager execution a spin. See getting started and the programmer's guide.

Hope that helps.

Upvotes: 1

Related Questions