CODEWITHSUNDEEP
pythontensorflowmachine-learningneural-networklstm
Talal Zahid
Talal Zahid

Reputation: 128

LSTM Neural Network Input/Output dimensions error

I am fairly new to TensorFlow and LSTM architecture. I have an issue with figuring out input and output (x_train, x_test, y_train, y_test) for my dataset.

Original shapes of my inputs:

The y_train and y_test are a series of stock prices. The x_train and x_test are four features that I want to use to predict the stock prices.

# Splitting the training and testing data

train_start_date = '2010-01-08'
train_end_date = '2017-01-06'
test_start_date = '2017-01-13'
test_end_date = '2019-01-04'

train = df.ix[train_start_date : train_end_date]
test = df.ix[test_start_date:test_end_date]


X_test = sentimentScorer(test)
X_train = sentimentScorer(train)

Y_test = test['prices'] 
Y_train = train['prices']

#Conversion in 3D array for LSTM INPUT

X_test = X_test.reshape(1, 104, 4)
X_train = X_train.reshape(1, 366, 4)





model = Sequential()

model.add(LSTM(128, input_shape=(366,4), activation='relu', 
return_sequences=True))
model.add(Dropout(0.2))

model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.1))

model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))

model.add(Dense(10, activation='softmax'))

opt = tf.keras.optimizers.Adam(lr=0.001, decay=1e-6)

# Compile model
model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer=opt,
    metrics=['accuracy'],
)

model.fit(X_train,
          Y_train,
          epochs=3,
          validation_data=(X_test, Y_test))

This is the error generated:

> --------------------------------------------------------------------------- ValueError                                Traceback (most recent call
> last) <ipython-input-101-fd4099583529> in <module>
>      65           Y_train,
>      66           epochs=3,
> ---> 67           validation_data=(X_test, Y_test))
> 
> c:\users\talal\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\training.py
> in fit(self, x, y, batch_size, epochs, verbose, callbacks,
> validation_split, validation_data, shuffle, class_weight,
> sample_weight, initial_epoch, steps_per_epoch, validation_steps,
> **kwargs)    1507         steps_name='steps_per_epoch',    1508         steps=steps_per_epoch,
> -> 1509         validation_split=validation_split)    1510     1511     # Prepare validation data.
> 
> c:\users\talal\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\training.py
> in _standardize_user_data(self, x, y, sample_weight, class_weight,
> batch_size, check_steps, steps_name, steps, validation_split)
>     991       x, y = next_element
>     992     x, y, sample_weights = self._standardize_weights(x, y, sample_weight,
> --> 993                                                      class_weight, batch_size)
>     994     return x, y, sample_weights
>     995 
> 
> c:\users\talal\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\training.py
> in _standardize_weights(self, x, y, sample_weight, class_weight,
> batch_size)    1110         feed_input_shapes,    1111        
> check_batch_axis=False,  # Don't enforce the batch size.
> -> 1112         exception_prefix='input')    1113     1114     if y is not None:
> 
> c:\users\talal\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\training_utils.py
> in standardize_input_data(data, names, shapes, check_batch_axis,
> exception_prefix)
>     314                            ': expected ' + names[i] + ' to have ' +
>     315                            str(len(shape)) + ' dimensions, but got array '
> --> 316                            'with shape ' + str(data_shape))
>     317         if not check_batch_axis:
>     318           data_shape = data_shape[1:]
> 
> ValueError: Error when checking input: expected lstm_18_input to have
> 3 dimensions, but got array with shape (366, 4)

Upvotes: 4

Views: 1752

Answers (3)

Szymon Maszke
Szymon Maszke

Reputation: 24894

Your code is almost fine.

Your y_test and y_train should be an array with one element or array of shape (1,1), it doesn't matter.

Your input shape is wrong though, first LSTM should be:

model.add(LSTM(128, input_shape=(None,4), activation='relu', return_sequences=True))

Notice None, as your test and train sequences length are different, you cannot specify it (and Keras accepts first dimension unspecified). Error was due to lengths of 366 and 104 respectively. If you want to use batches with RNNs you should perform zero-padding with keras.preprocessing.sequence.pad_sequences.

No need to specify input_shape with batch, rest of the network should be fine.

And if you are performing regression, not classification, as is the case probably, you should perform two last steps written by @Ankish Bansal, e.g. changing loss to mean squared error and making last layer output 1 value instead of 10.

Upvotes: 5

Ankish Bansal
Ankish Bansal

Reputation: 1900

  1. LSTM expects input of dims as (num_examples, seq_length, input_dims), so there is one error in input.

  2. You are predicting output of dimension 1, what your model output is 10. Try this.

    model.add(Dense(1, activation='linear'))

  3. Also, you are predicting price, that is regression problem. But, you are using classification setting. Try this

    model.compile(loss='mse', optimizer='adam', metrics=['mean_squared_error'])

Upvotes: 3

Dulmina
Dulmina

Reputation: 527

your dimension is wrong for X_train.LSTM only accepts 3-dimensional inputs.you are saying that you have 4 features. Assuming that 366 is your number of timestamps for one sample, your input should be in (num_samples,366,4) shape cheers :-)

Upvotes: 1

Related Questions