CODEWITHSUNDEEP
machine-learningtensorflowlstmrecurrent-neural-network
Tobi
Tobi

Reputation: 93

How can I change the max sequence length in a Tensorflow RNN Model?

I am currently trying to adapt my tensorflow classifier, which is able to tag a sequence of words to be positive or negative, to handle much longer sequences, without retraining. My model is a RNN, with a max sequence lenght of 210. One input is one word(300 dim), I vectorised the words with Googles word2vec, so I am able to feed a sequence with max 210 words. Now my question is, how can I change the max sequence length to for example 3000, for classifying movie reviews.

My working model with fixed max sequence length of 210(tf_version: 1.1.0):

n_chunks = 210
chunk_size = 300

x = tf.placeholder("float",[None,n_chunks,chunk_size])
y = tf.placeholder("float",None)
seq_length = tf.placeholder("int64",None)


with tf.variable_scope("rnn1"):
        lstm_cell = tf.contrib.rnn.LSTMCell(rnn_size, 
                                            state_is_tuple=True)

        lstm_cell = tf.contrib.rnn.DropoutWrapper (lstm_cell, 
                                                   input_keep_prob=0.8)

        outputs, _ = tf.nn.dynamic_rnn(lstm_cell,x,dtype=tf.float32, 
                                       sequence_length = self.seq_length)

fc = tf.contrib.layers.fully_connected(outputs, 1000, 
                                      activation_fn=tf.nn.relu)

output = tf.contrib.layers.flatten(fc)

#*1
logits = tf.contrib.layers.fully_connected(output, self.n_classes, 
                                            activation_fn=None) 

cost = tf.reduce_mean( tf.nn.softmax_cross_entropy_with_logits 
                                        (logits=logits, labels=y) )
optimizer = tf.train.AdamOptimizer(learning_rate=0.01).minimize(cost)

...
#train
#train_x padded to fit(batch_size*n_chunks*chunk_size)
sess.run([optimizer, cost], feed_dict={x:train_x, y:train_y, 
                                                     seq_length:seq_length})
#predict:
...

pred = tf.nn.softmax(logits)
pred = sess.run(pred,feed_dict={x:word_vecs, seq_length:sq_l})

What modifications I already tried:

1 Replacing n_chunks with None and simply feed data in 

x = tf.placeholder(tf.float32, [None,None,300])
#model fails to build
#ValueError: The last dimension of the inputs to `Dense` should be defined. 
#Found `None`.
# at *1

...
#all entrys in word_vecs still have got the same length for example 
#3000(batch_size*3000(!= n_chunks)*300)
pred = tf.nn.softmax(logits)
pred = sess.run(pred,feed_dict={x:word_vecs, seq_length:sq_l})

2 Changing x and then restore the old model:

x = tf.placeholder(tf.float32, [None,n_chunks*10,chunk_size]
...
saver = tf.train.Saver(tf.all_variables(), reshape=True)
saver.restore(sess,"...")
#fails as well:
#InvalidArgumentError (see above for traceback): Input to reshape is a 
#tensor with 420000 values, but the requested shape has 840000
#[[Node: save/Reshape_5 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, 
#_device="/job:localhost/replica:0/task:0/cpu:0"](save/RestoreV2_5, 
#save/Reshape_5/shape)]]

# run prediction

If it is possible could you please provide me with any working example or explain me why it isnt?

Upvotes: 0

Views: 1195

Answers (1)

Lerner Zhang
Lerner Zhang

Reputation: 7150

I am just wondering why not you just assign the n_chunk a value of 3000?

In your first attempt, you cannot use two None, since tf cannot how many dimensions to put for each one. The first dimension is set as None because it is contingent upon the batch size. In your second attempt, you just change one place and the other places where n_chunks is used may conflict with the x placeholder.

Upvotes: 1

Related Questions