CODEWITHSUNDEEP
pythontensorflowneural-networkkeraslstm
djokester
djokester

Reputation: 586

Sequence to Sequence Learning with Keras

I was trying to implement a sequence to sequence learning task in Keras. I want the model to learn the following transformation. F(X) -> Y Where X is 

  [[0., 0., 1., 0., 1., 1., 0., 0., 0., 0.],
   [1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
   [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
   [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
   [0., 1., 0., 1., 0., 0., 0., 0., 0., 0.],
   [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
   [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
   [1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
   [0., 0., 0., 0., 0., 1., 0., 0., 0., 0.],
   [0., 0., 0., 0., 1., 0., 0., 0., 0., 0.]]

and Y is given by 

  [[1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
   [0., 0., 0., 0., 0., 1., 0., 0., 0., 0.],
   [0., 0., 0., 0., 1., 0., 0., 0., 0., 0.],
   [0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
   [0., 1., 0., 1., 0., 0., 0., 0., 0., 0.]]

Each column in the X and Y array represents a word in a sentence. However, each word has ten features in the input sequence (i.e X) and just five features in Y. I tried implementing a model with the following code. 

model = Sequential()
LAYER_NUM = 3
HIDDEN_DIM = 900
model = Sequential()
model.add(Bidirectional(LSTM(HIDDEN_DIM, return_sequences=True), input_shape = (None, 10)))
print("YEAH!")
for i in range(LAYER_NUM - 1):
    model.add(Bidirectional(LSTM(HIDDEN_DIM, return_sequences=True)))
    model.add(Dropout(0.2))
model.add(TimeDistributed(Dense(5)))
model.add(Activation('softmax'))
model.compile(loss="categorical_crossentropy", optimizer="rmsprop", metrics=["accuracy"])

However, this raised the following error.

ValueError: Error when checking target: expected activation_3 to have shape (10, 5) but got array with shape (5, 10)

Can someone guide me on this?

Upvotes: 0

Views: 148

Answers (1)

nuric
nuric

Reputation: 11225

The error gives it away. You said columns represent the words, in the Keras model you have it would be rows. So you have to transpose your data X.T and Y.T to fix it.

Upvotes: 1

Related Questions