CODEWITHSUNDEEP
pythonkeraslstm
Shamoon
Shamoon

Reputation: 43609

How to fix Keras LSTM input / output dimensions?

My model is:

model = Sequential()
model.add(Embedding(input_dim=vocab_size,
                    output_dim=1024, input_length=self.SEQ_LENGTH))

model.add(LSTM(vocab_size))

model.add(Dropout(rate=0.5))
model.add(Dense(vocab_size, activation='softmax'))
print(model.summary())

model.compile(loss='sparse_categorical_crossentropy',
              optimizer="adam", metrics=['accuracy'], callbacks=callbacks)

The summary is:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, 100, 1024)         5064704   
_________________________________________________________________
lstm_1 (LSTM)                (None, 4946)              118130264 
_________________________________________________________________
dropout_1 (Dropout)          (None, 4946)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 4946)              24467862  
=================================================================
Total params: 147,662,830
Trainable params: 147,662,830
Non-trainable params: 0
_________________________________________________________________

But when I run it, I get an error:

ValueError: Error when checking target: expected dense_1 to have shape (1,) but got array with shape (4945,)

What do I need to change to get it to match better?

Upvotes: 0

Views: 39

Answers (1)

ixeption
ixeption

Reputation: 2060

I don't think your model is what you want:

model.add(LSTM(vocab_size))

You don't want a LSTM layer with 4946 neurons.

model.add(Dense(vocab_size, activation='softmax'))

And I guess you don't want to predict 4946 classes.

but got array with shape (4945,)

You model expects a dataset, where each row is a list of word indices, where the max index is vocab_size.

What do I need to change to get it to match better? I think you should take a look at the keras NLP examples. It pretty well explained.

Upvotes: 1

Related Questions