Keras Financial Neural Network Input Error: Expected 4 Dimensions, Received Input Shape (1172, 1, 5)

Question

This is the code I've written to predict stock market fluctuations in historical Facebook data. I am using a Keras neural network and the data is sourced from Quandl. The program utilises information derived from the historical finance database previously referenced to train a neural network within the prediction of stock prices, with derived components from posts authored via Michael Grogan (MGCodesAndStats). The program is below:

import tensorflow as tf
import keras
import numpy as np
import quandl
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import pandas as pd
import sklearn
import math

df = quandl.get("WIKI/FB", api_key = '_msxC6xspj2ddytz7-4u')
print(df)

df = df[['Adj. Close']]

previous = 1

def create_dataset(df, previous):
    dataX, dataY = [], []
    for i in range(len(df)-previous-1):
        a = df[i:(i+previous), 0]
        dataX.append(a)
        dataY.append(df[i + previous, 0])
    return np.array(dataX), np.array(dataY)

scaler = sklearn.preprocessing.MinMaxScaler(feature_range=(0, 1))
df = scaler.fit_transform(df)
print(df)

train_size = math.ceil(len(df) * 0.8)

train, val = df[0:train_size,:], df[train_size:len(df),:]

X_train, Y_train = create_dataset(train, previous)

print(X_train)
print(Y_train)

X_val, Y_val = create_dataset(val, previous)

X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
X_val = np.reshape(X_val, (X_val.shape[0], 1, X_val.shape[1]))

model = keras.models.Sequential() 
model.add(keras.layers.Dense(units = 64, activation = 'relu', input_shape = (1172, 1, 5)))
model.add(keras.layers.Dense(units = 1, activation = 'linear'))
model.compile(loss='mean_absolute_error', 
              optimizer='adam', 
              metrics=['accuracy'])

model.fit(X_train, Y_train, epochs=8)

However, despite the fact that the shape of the information provided to the neural network is specified, the program produced the following error message:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-d11b5f4c50ab> in <module>()
     53               metrics=['accuracy'])
     54 
---> 55 model.fit(X_train, Y_train, epochs=8)
     56 
     57 

2 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
    129                         ': expected ' + names[i] + ' to have ' +
    130                         str(len(shape)) + ' dimensions, but got array '
--> 131                         'with shape ' + str(data_shape))
    132                 if not check_batch_axis:
    133                     data_shape = data_shape[1:]

ValueError: Error when checking input: expected dense_1_input to have 4 dimensions, but got array with shape (1172, 1, 5)

This error additionally occurs despite the fact that the shape of the array to remain provided to the first layer of the neural network is specified within its entirety as possessing a shape of (1172, 1, 5), the shape which the program states is not expected; is there an immediate manner via which this problem may effortlessly remain solved? What is the primary cause of the error, with the input shape specified? Thank you for your assistance.

Answer 1

You are making the classic error of including the samples dimension in the input shape, which is if course not correct, your input shape should be:

model.add(keras.layers.Dense(units = 64, activation = 'relu', input_shape = (1, 5)))