Keras LSTM Model not learning

Question

I wrote this code a few days ago and I had a few bugs but with some help, I was able to fix them. The Model is not learning. I tried different batch sizes, different amount of epochs, different activation functions, checked my data a few times for flaws I wasn't able to find any. It is due in a week or so for a school project. Any help will be very much valued.

Here is the code.

from keras.layers import Dense, Input, Concatenate, Dropout
from sklearn.preprocessing import MinMaxScaler
from keras.models import Model
from keras.layers import LSTM
import tensorflow as tf
import NetworkRequest as NR
import ParseNetworkRequest as PNR
import numpy as np


def buildModel():
    _Price = Input(shape=(1, 1))
    _Volume = Input(shape=(1, 1))
    PriceLayer = LSTM(128)(_Price)
    VolumeLayer = LSTM(128)(_Volume)
    merged = Concatenate(axis=1)([PriceLayer, VolumeLayer])
    Dropout(0.2)
    dense1 = Dense(128, input_dim=2, activation='relu', use_bias=True)(merged)
    Dropout(0.2)
    dense2 = Dense(64, input_dim=2, activation='relu', use_bias=True)(dense1)
    Dropout(0.2)
    output = Dense(1, activation='softmax', use_bias=True)(dense2)

    opt = tf.keras.optimizers.Adam(learning_rate=1e-3, decay=1e-6)

    _Model = Model(inputs=[_Price, _Volume], output=output)
    _Model.compile(optimizer=opt, loss='mse', metrics=['accuracy'])

    return _Model


if __name__ == '__main__':
    api_key = "47BGPYJPFN4CEC20"
    stock = "DJI"
    Index = ['4. close', '5. volume']

    RawData = NR.Initial_Network_Request(api_key, stock)

    Closing = PNR.Parse_Network_Request(RawData, Index[0])
    Volume = PNR.Parse_Network_Request(RawData, Index[1])
    Length = len(Closing)

    scalar = MinMaxScaler(feature_range=(0, 1))

    Closing_scaled = scalar.fit_transform(np.reshape(Closing[:-1], (-1, 1)))
    Volume_scaled = scalar.fit_transform(np.reshape(Volume[:-1], (-1, 1)))
    Labels_scaled = scalar.fit_transform(np.reshape(Closing[1:], (-1, 1)))

    Train_Closing = Closing_scaled[:int(0.9 * Length)]
    Train_Closing = np.reshape(Train_Closing, (Train_Closing.shape[0], 1, 1))

    Train_Volume = Volume_scaled[:int(0.9 * Length)]
    Train_Volume = np.reshape(Train_Volume, (Train_Volume.shape[0], 1, 1))

    Train_Labels = Labels_scaled[:int((0.9 * Length))]
    Train_Labels = np.reshape(Train_Labels, (Train_Labels.shape[0], 1))

    # -------------------------------------------------------------------------------------------#

    Test_Closing = Closing_scaled[int(0.9 * Length):(Length - 1)]
    Test_Closing = np.reshape(Test_Closing, (Test_Closing.shape[0], 1, 1))

    Test_Volume = Volume_scaled[int(0.9 * Length):(Length - 1)]
    Test_Volume = np.reshape(Test_Volume, (Test_Volume.shape[0], 1, 1))

    Test_Labels = Labels_scaled[int(0.9 * Length):(Length - 1)]
    Test_Labels = np.reshape(Test_Labels, (Test_Labels.shape[0], 1))

    Predict_Closing = Closing_scaled[-1]
    Predict_Closing = np.reshape(Predict_Closing, (Predict_Closing.shape[0], 1, 1))

    Predict_Volume = Volume_scaled[-1]
    Predict_Volume = np.reshape(Predict_Volume, (Predict_Volume.shape[0], 1, 1))

    Predict_Label = Labels_scaled[-1]
    Predict_Label = np.reshape(Predict_Label, (Predict_Label.shape[0], 1))

    model = buildModel()
    model.fit(
        [
            Train_Closing,
            Train_Volume
        ],
        [
            Train_Labels
        ],
        validation_data=(
            [
                Test_Closing,
                Test_Volume
            ],
            [
                Test_Labels
            ]
        ),
        epochs=10,
        batch_size=Length
    )

This is the output when I run it.

Using TensorFlow backend.
2020-01-01 16:31:47.905012: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2199985000 Hz
2020-01-01 16:31:47.906105: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x49214f0 executing computations on platform Host. Devices:
2020-01-01 16:31:47.906137: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
/home/martin/PycharmProjects/MarketPredictor/Model.py:26: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=[



The loss is high, and the accuracy is 0.
Please help.

Nicolas Gervais · Accepted Answer

You're using activation functions and metrics made for a classification task, not a stock forecasting task (with a continuous target).

For continuous targets, your final activation layer should be linear. Metrics should be mse or mae, not accuracy.

accuracy would only be satisfied is the dji prediction is exactly equal to the actual price. Since dji has at least 7 digits, it's nearly impossible.

Keras LSTM Model not learning

Answers (2)

Related Questions