CODEWITHSUNDEEP
neural-networkkeras
Michael
Michael

Reputation: 3366

Recurrent neural layers in Keras

I'm learning neural networks through Keras and would like to explore my sequential dataset on a recurrent neural network. I was reading the docs and trying to make sense of the LSTM example.

My questions are:

  1. What are the timesteps that are required for both layers?
  2. How do I prepare a sequential dataset that works with Dense as an input for those recurrent layers?
  3. What does the Embedding layer do?

Upvotes: 6

Views: 1189

Answers (1)

Marcin Możejko
Marcin Możejko

Reputation: 40516

  1. Timesteps are a pretty bothering thing about Keras. Due to the fact that data you provide as an input to your LSTM must be a numpy array it is needed (at least for Keras version <= 0.3.3) to have a specified shape of data - even with a "time" dimension. You can only put a sequences which have a specified length as an input - and in case your inputs vary in a length - you should use either an artificial data to "fill" your sequences or use a "stateful" mode (please read carefully Keras documentation to understand what this approach means). Both solutions might be unpleasent - but it's a cost you pay that Keras is so simple :) I hope that in version 1.0.0 they will do something with that.

  2. There are two ways to apply norecurrent layers after LSTM ones:

    • you could set an argument return_sequences to False - then only the last activations from every sequence will be passed to a "static" layer.
    • you could use one of "time distributed" layers - to get more flexibility with what you want to do with your data.

  3. https://stats.stackexchange.com/questions/182775/what-is-an-embedding-layer-in-a-neural-network :)

Upvotes: 1

Related Questions