CODEWITHSUNDEEP
pythondeep-learningkeraslstm
sachinruk
sachinruk

Reputation: 9869

One to One LSTM

I have a sequence of numbers of length 1000, which I chop up into 100 length sequences. So I end up with 901 sequences of length 100. Let the first 900 be trainX. trainY is the 2nd to the 901th of these sequences.

In keras I wish to build the following pictured model: enter image description here The important features are that $X_1$ maps to $X_2$, $X_2$ maps to $X_3$ and so on. Ignore the fact that I haven't drawn 100 units of these.

In keras I know how to build the model where $X_1$ to $X_{100}$ maps to $X_{101}$ (many to one case). Which is done by the following:

batch_size = 1
time_steps = 100
model = Sequential()
model.add(LSTM(32, batch_input_shape=(batch_size, time_steps, 1), stateful=True))
model.add(Dense(1))

However, in the many to many case, the following chucks an error:

model = Sequential()
model.add(LSTM(32, batch_input_shape=(batch_size, time_steps, 1), stateful=True, return_sequences=True))
model.add(Dense(1))

I try to preserve the fact that 100 outputs are given by saying return_sequences=True. I get the error Input 0 is incompatible with layer dense_6: expected ndim=2, found ndim=3, which I guess is understandable given that Dense only expects a batch_size x hidden_nodes size matrix as input whereas, in my case it outputs, batch_size x time_steps x hidden_nodes.

So my question is how do I get a LSTM that behaves as shown in the picture. It is important that in the dense layer I do not accidentally reference a hidden layer in front (or back for that matter) of the current time step.

Upvotes: 1

Views: 1369

Answers (1)

Nejla
Nejla

Reputation: 133

You don't need multiple outputs. Train your model to predict the next number in the sequence. Then, use the predicted number and feed it to the trained model and predict the next one and repeat this task. In other words:

Prepare data for training:

X_train = np.zeros((901, 100))
y_train = np.zeros((901, 1))
for i in range(901)
  x_train[i,:] = x[i:i+100]
  y_train[i,0] = x[i+101]

Train your model:

for iteration in range(10000):
  model.fit(x_train, y_train, batch_size = 40, nb_epoch = 2)

Now, if you want to predict the next 10 numbers starting with: x[t: t+101]

all you need to do is:

    x_input = np.zeros((1, 100))
    x_input[0, :] =  x[t+i: t+i+101]
    for i in range(10)
      y_output = model.predict(x_input)
      print(y_output)
      x_input[0, 0:100] =  x[t+i+1: t+i+100]
      x_input[100] = y_output

I used batch_size = 40 in this example. But, you can use anything (but I don't recommend 1! ;))

Upvotes: 1

Related Questions