CODEWITHSUNDEEP
tensorflow
minorfox
minorfox

Reputation: 23

ValueError: Trying to share variable enco/gru_cell/gates/kernel, but specified shape (512, 512) and found shape (356, 512)

I encounter an error after I updated my tensorflow-GPU to 1.2.1. I used encode-decode structure, part of encode's code is:

output, state = tf.nn.dynamic_rnn(cell=self.cell,
                              inputs=logits,
                              sequence_length=self.en_len,
                              dtype=tf.float32,
                              scope='enco')

and the decode is:

output, _ = tf.nn.dynamic_rnn(cell=self.cell,
                                      inputs=deco_inputs,
                                      sequence_length=self.de_len,
                                      initial_state=init_state,
                                      dtype=tf.float32,
                                      scope='deco')

then, I received the error. Num of hidden units is 256, embeding size is 100, to check the error, I removed multiRNN and used single GRU cell with:

RNN = tf.contrib.rnn.GRUCell(HIDDEN_UNITs)

Here is the full error:

ValueError: Trying to share variable enco/gru_cell/gates/kernel, but specified shape (512, 512) and found shape (356, 512).

ps. This error never happened in version 1.0.1.

Upvotes: 2

Views: 1067

Answers (1)

javidcf
javidcf

Reputation: 59731

This problem usually appears only when you use MultiRNNCell, but in your case it happens even with a single cell because you have two RNNs. In short, starting from TensorFlow 1.2 RNN cells should not be reused for different layers of an RNN or for different RNNs. If you want to use a single GRUCell for each RNN, you need to call the constructor twice to have two different instances to pass. If you are using MultiRNNCells you should have as well two of these, but also one instance of GRUCell per layer on each of these. So:

# This is doubly wrong!
# self.cell = tf.contrib.rnn.MultiRNNCell(
#     [tf.contrib.rnn.GRUCell(HIDDEN_UNITs)] * NUM_LAYERS)

# Also wrong
# self.cell_encoder = tf.contrib.rnn.MultiRNNCell(
#     [tf.contrib.rnn.GRUCell(HIDDEN_UNITs)] * NUM_LAYERS)
# self.cell_decoder = tf.contrib.rnn.MultiRNNCell(
#     [tf.contrib.rnn.GRUCell(HIDDEN_UNITs)] * NUM_LAYERS)

# Still wrong
# self.cell = tf.contrib.rnn.MultiRNNCell(
#     [tf.contrib.rnn.GRUCell(HIDDEN_UNITs) for _ in range(NUM_LAYERS)])

# This is the way to go!
self.cell_encoder = tf.contrib.rnn.MultiRNNCell(
    [tf.contrib.rnn.GRUCell(HIDDEN_UNITs) for _ in range(NUM_LAYERS)])
self.cell_decoder = tf.contrib.rnn.MultiRNNCell(
    [tf.contrib.rnn.GRUCell(HIDDEN_UNITs) for _ in range(NUM_LAYERS)])

And then you build your RNNs:

output_encoder, state_encoder = \
    tf.nn.dynamic_rnn(cell=self.cell_encoder,
                      inputs=logits,
                      sequence_length=self.en_len,
                      dtype=tf.float32,
                      scope='enco')
output_decoder, _ = tf.nn.dynamic_rnn(cell=self.cell_decoder,
                                      inputs=deco_inputs,
                                      sequence_length=self.de_len,
                                      initial_state=init_state,
                                      dtype=tf.float32,
                                      scope='deco')

Upvotes: 3

Related Questions