CODEWITHSUNDEEP
tensorflowkeraslstm
neo33
neo33

Reputation: 1879

how to add more layers to the following keras architecture?

I am working on a keras model using tensorflow as a back-end:

model = Sequential() 
model.add(Embedding(40, 64, dropout=0.2))
model.add(LSTM(33, dropout_W=0.2, dropout_U=0.2))  
model.summary()
model.add(Dense(2))
model.add(Activation('relu'))
model.compile(loss='binary_crossentropy', optimizer='sgd', metrics=['binary_accuracy'])
print('sumary model')
model.summary()

In fact this architecture is working very well as a binary classifier of sentences, however I would like to improve this since it is a little simple. I am a begginer using keras, so I would like to appreciate suggestions to modify it. I would like to add more layers and dropout factors to get better results and the main idea of my post is to receive suggestions.

Upvotes: 0

Views: 1772

Answers (1)

mikkokotila
mikkokotila

Reputation: 1431

The best way to learn about adding layers to Keras models, is by running and studying the various examples provided in the Keras repo.

The model code that you had shared above looks a little bit random to be honest. But if you have a well working model, and want to make it work better, generally speaking it's better to tune the layers you already have, rather than add new ones. Simple is better than complex. Make no mistake. Rather than add more complexity in terms of the structure of the network, try to make what you have already work better. You could:

  • set output activation to 'sigmoid'
  • change your optimizer to something else than SGD
  • try different number of embed input_dims
  • change the learning rate of the optimizer

Other than that, the LSTM layer has roughly 20 parameters, check out the docstring to see what is there.

If you must add more layers, then you can try first just Dense layers. Something like: 

model.add(Dense(no_of_neurons))
model.add(Dropout(dropout_rate))

Also I don't think there is any reason to call model.summary() twice as you do now.

I think the key point is to keep thing as simple as you can. Keras is amazing that way; you can get great results with code that looks like it's nothing. If you're serius about learning Keras, I highly recommend reading Deep Learning with Python by Francois who is the guy behind Keras.

Upvotes: 1

Related Questions