CODEWITHSUNDEEP
pythonstatictime-seriesrecurrent-neural-network
Ali Wali
Ali Wali

Reputation: 339

How to combine static features with time series in forecasting

I tried to find a similar question and its answers but was not successful in doing so. That's why I'm asking a question that might be asked before:

I'm working on a problem that outputs the cumulative water production of several water wells. The features I have are both time series (water rate and pump speed as functions of time) and static (depth of the wells, latitude and longitude of the well, thickness of the water bearing zone, etc.)

My input data can be shown as below for well#1.

dynamic data:

                water rate   pump speed   total produced water
2000-01-01         10            4                 1120
2000-01-02         20            8                 1140
2000-01-03         10            4                 1150
2000-01-04         10            3                 1160
2000-01-05         10            4                 1170

static data:

depth of the well_1 = 100
latitude and longitude of the well_1 = x1, y1
thickness of the water bearing zone of well_1 = 3

My question is how a RNN model (LSTM, GRU, ...) can be built that can take both dynamic and static features?

Upvotes: 8

Views: 5885

Answers (2)

Qinqing Liu
Qinqing Liu

Reputation: 432

LSTM_att proposed by Machine Learning Crop Yield Models Based on Meteorological Features and Comparison with a Process-Based Modelenter link description here seems to be a good option.
It applies static features to calculate the attention to aggregate the hidden states of time series and also provides a shortcut connection between each hidden state and final state (similar to ResNet). It outperforms baseline LSTM models.

Upvotes: 1

Zabir Al Nazi Nabil
Zabir Al Nazi Nabil

Reputation: 11218

There are multiple options, and you need to experiment which one will be optimal for your case.

Option 1: You can treat your static features as fixed temporal data. So, you make a temporal dimension for each of your static features and let LSTM handle the rest.

For example your transformed data will look like this:

                water rate   pump speed   total produced water   depth_wall
2000-01-01         10            4                 1120             100
2000-01-02         20            8                 1140             100
2000-01-03         10            4                 1150             100
2000-01-04         10            3                 1160             100
2000-01-05         10            4                 1170             100

Option 2: Designing multi-head networks.

TIME_SERIES_INPUT ------> LSTM -------\
                                       *---> MERGE / Concatenate ---> [more layers]
STATIC_INPUTS --> [FC layer/ conv] ---/

Here is a paper explaining a combining strategy: https://arxiv.org/pdf/1712.08160.pdf

Here is another paper utilizing option 2: https://www.researchgate.net/publication/337159046_Classification_of_ECG_signals_by_dot_Residual_LSTM_Network_with_data_augmentation_for_anomaly_detection

Source code for paper 2: https://github.com/zabir-nabil/dot-res-lstm

Upvotes: 9

Related Questions