CODEWITHSUNDEEP
pythontensorflowmachine-learninglstm
Bowers
Bowers

Reputation: 846

LSTM network for daily weather forecast with init state

For the forecast of two weather values, I have a dataset that consists of one year data with a fifteen minute resolution. So data for 96 time points per day and 365 * 96 in total are available.

Whenever I enter a day of data in the LSTM network for training, I also want to give real measured data for the output state h.

# tensor for setting output state h
init_state = tf.reshape(init_state, [1, 2])
init_state = tf.nn.rnn_cell.LSTMStateTuple(tf.constant([[0, 0]], tf.float32), init_state)

# tensor placeholder for input data with 2 inputs
train_placeholder = tf.placeholder(tf.float32, [1, 96, 2])

# hidden size equals 2, because LSTM output and output state h have 2 values
cell = tf.contrib.rnn.LSTMCell(2)
rnn_outputs_ts, state = tf.nn.dynamic_rnn(cell, train_placeholder,
                                          dtype=tf.float32, initial_state=init_state)

The training takes place on a daily basis in a loop:

# daily training of the weather data
for step in len(train_data):
    sess.run(train_lstm, {train_placeholder: train_data[step].reshape(1, 96, 2),
                          init_state: init_data[step].reshape(1, 2) })

If I set the output state h in each iteration as described, I get better results than if the output state h is always set to zero by default.

Since the output state h contains two values for which I enter real measured values, the hidden size of the LSTM cell is also limited to 2. However, without manually setting the output state h, I noticed that if the hidden size of the LSTM cell is larger than 2 and the outputs get flatten to 2, much better results are possible. The same is true for MultiRNNCell, which I also don't use because of the output state h.

On the one hand I want a larger LSTM network, but on the other hand I also want to set the output state h with real measurement data.

How would you proceed in my case?

Upvotes: 2

Views: 160

Answers (1)

Kai Aeberli
Kai Aeberli

Reputation: 1220

I think you need another layer - networks usually have an input layer, a number intermediate layers, and an output layer that does the final classification. In your case, you could add an Dense() layer with 2 neurons after the LSTM layer.

Also, LSTMCell is deprecated (https://www.tensorflow.org/api_docs/python/tf/nn/rnn_cell/LSTMCell) and will be removed in tensorflow 2.0. I would suggest to use Keras (a wrapper around tensorflow) to setup your network - it provides many convenience functions and helps analyse your model. Check here for an explanation how to setup an LSTM model: https://adventuresinmachinelearning.com/keras-lstm-tutorial/

Upvotes: 2

Related Questions