CODEWITHSUNDEEP
recurrent-neural-networktensorflow2.0encoderseq2seq
wyh127
wyh127

Reputation: 23

how to build a custom bidirectional encoder for seq2seq with tf2?

class Encoder(tf.keras.Model):
  def __init__(self, vocab_size, embedding_dim, enc_units, batch_sz):
    super(Encoder, self).__init__()
    self.batch_sz = batch_sz
    self.enc_units = enc_units

    self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)
    self.gru = tf.keras.layers.GRU(self.enc_units,
                                   return_sequences=True,
                                   return_state=True,
                                   recurrent_initializer='glorot_uniform')

    self.bigru=tf.keras.layers.Bidirectional(tf.keras.layers.GRU(self.enc_units,
                                                                 return_sequences=True,
                                                                 return_state=True, recurrent_initializer='glorot_uniform'))

  def call(self, x):
    x = self.embedding(x)
    # output, state = self.gru(x)
    output, state = self.bigru(x)

    return output, state

For the above code, when I used the gru layer, it worked. But when I used the bigru layer, I got the following error:

ValueError: in converted code:

<ipython-input-51-3ba1fe0beb05>:8 train_step_seq2seq  *
    enc_output, enc_hidden = encoder(inp)
/tensorflow-2.0.0/python3.6/tensorflow_core/python/keras/engine/base_layer.py:847 __call__
    outputs = call_fn(cast_inputs, *args, **kwargs)
<ipython-input-53-4f1b00e47a9a>:22 call  *
    output, state = self.bidir(x)

ValueError: too many values to unpack (expected 2)

So I'm now wondering what is going on here?

Upvotes: 2

Views: 702

Answers (1)

Jindřich
Jindřich

Reputation: 11230

It is not well-documented, but the bidirectional layer (unlike the single direction RNN layer) returns a three-tuple:

  • concatenated states of the forward and backward RNN (shape: batch × length × 2 GRU dimentsion)
  • final state of the forward RNN (shape: batch × batch dimension)
  • final state of the backward RNN (shape: batch × batch dimension)

Upvotes: 1

Related Questions