CODEWITHSUNDEEP
pythonkeraslstmfeature-extraction
Aaron2Dev
Aaron2Dev

Reputation: 61

Feature extraction for Timeseries LSTM

I want to feed a timeseries into an LSTM to perform a forecast. Lets say I have 10000 samples. Now in order to feed the timeseries into my LSTM I reshape it to (samples,timesteps,features). In my case I use timesteps=50 to create subsequences and perform a forecast of t+1. So I end up with x.shape=(9950,50,1). So far so good.

My Model

model= Sequential()
model.add(LSTM(50,return_sequences=True,input_shape=(50,1)))
model.add(Dense(out_dim, activation = 'sigmoid'))
model.compile(loss='mse', optimizer='adam')

NOW I want to create artificial features e.g. I want to use the fft of the signal as a feature. How I can feed it into my LSTM? Is it legitimate to just compute the fft, append it to the Dataframe and reshape all together so I end up with (9950,50,2)??

Questions are basically:

  1. How I input artifical created features into an LSTM?
  2. Its the same way for rolling statistics or correlation features?

Thanks in advance

Upvotes: 1

Views: 4921

Answers (1)

nuric
nuric

Reputation: 11225

Any extra feature you compute from the input data is just another feature so:

  1. You feed it just like another feature of series, input_shape=(50, 1+extra_features) and you will have to concatenate those prior to passing to model. So yes, the input shape will now be (9950, 50, 2).
  2. Yes it is, you can pre-compute that feature let's say moving average and then concatenate it with the original input.

You can also write custom layers to compute these features within the model, but the model would have compute it every time. If you compute it a priori, the advantage is you can save / cache it.

If you have non-timeseries features, now you need to move onto the functional API and have multiple inputs: 1 which is timeseries and another which is not:

series_in = Input(shape=(50, 2))
other_in = Input(shape(extra_features,)) # not a timeseries just a vector
# An example graph
lstm_out = LSTM(128)(series_in)
merged = concatenate([lstm_out, other_in])
out = Dense(out_dim, activation='sigmoid')(merged)
model = Model([series_in, other_in], out)
model.compile(...)

In this case we have 2 inputs to the model and can use the auxiliary features at any point. In the example, I merge before the final Dense layer to aid the predication along with the timeseries features extracted with the LSTM.

Upvotes: 1

Related Questions