CODEWITHSUNDEEP
keraslstm
jwm
jwm

Reputation: 5040

Keras: the difference input_dim and input_length in LSTM

In building LSTM, we are required to provide input shape information by:

input_shape = () # a tuple

alternatively, by:

input_length = () # an integer
input_dim = () # an integer

I feel a little confused about the two quantities. What are they indicating?

Also, is the input_dim the so-called time steps?

Upvotes: 11

Views: 4683

Answers (2)

Vikas Sharma
Vikas Sharma

Reputation: 461

We can understand this in two scenarios, time series data and NLP.

In case of time series data, suppose we have the below dataset, that is day wise temperature for two weeks.

[23.4, 23.4,22.4,23.4,26.4,24.4,27.4,28.4,21.4,25.4,23.4,23.4,23.4, 24.5]

We want to take 5 day temperature and want to predict 6th day temperature. In this case the input_length = () would be 5 (ex. 23.4, 23.4,22.4,23.4,26.4) and input_dim = () would be 1 (value of temperature for each day).

In case of NLP, suppose sentence length is 4 for example "how are you doing" and output of our embedding layer is 8, than each word of sentence would be represented by vector of length 8, so "hello" would be vector of length 8, "are" would be vector or length 8 and so on. for this scenario input_length = () would be 4 (sentence length) and input_dim =() would be 8 (length of embedding vector).

Refer https://www.tensorflow.org/tutorials/structured_data/time_series for detailed information on LSTM for time series data.

Upvotes: 3

Huzaifa Calcuttawala
Huzaifa Calcuttawala

Reputation: 99

I will try to simplify the input shape parameters as much as possible: In case of LSTM (or in general for RNNs) the input shape can be provided either by:

  1. input_shape key word argument: input_shape = (input_length, input_dim) where input_length = length of sequence and input_dim = number of features/variables. If the values are not supplied, it indicates any positive integer can be expected. Here you do not mention the batch size i.e. number of observations for updates in weights during training. e.g input_length = 50 (this is your sequence length) input_dim = 10 (this is your number of input features in the data)

    model.add(LSTM(16, input_shape = (50,10)))

  2. using separate input_dim and input_length arguments Here you specify input_dim i.e. number of features in the data and input_length i.e. sequence length of single observation in your data separately. for e.g. model.add(LSTM(16, input_length= 50, input_dim =10)). This is equivalent to the way described above

  3. Lastly, you can specify the size of batch (which has not been specified above) through batch_input_size argument.if the LSTM is stateful you have to pre-specify the batch size. batch_input_size = (batch_size,input_length, input_dim)

model.add(LSTM(16,batch_input_size = (None,50,10))) equivalent to the above two

model.add(LSTM(16,batch_input_size = (32,50,10))) the batch size is 32

Upvotes: 4

Related Questions