CODEWITHSUNDEEP
tensorflowmachine-learningkerasdeep-learning
agnivesh
agnivesh

Reputation: 115

Keras reported accuracy doesn't correspond with model.predict()

Relevant portions of my code:

Model Architecture:

Sequential(
    layers=[
        BatchNormalization(input_shape=input_shape),
        LSTM(lstm_1, return_sequences=True, dropout=dropout_1),
        BatchNormalization(),
        LSTM(lstm_2, dropout=dropout_2),
        Dense(1, activation='sigmoid')
    ]
)

Compile and fit calls:

model.compile(loss='binary_crossentropy',
                            optimizer=RMSprop(0.0005),
                            metrics=['accuracy'])

model.fit(np.stack(data_train['spectrogram']), np.stack(data_train['label']),
          batch_size=512,
          epochs=50,
          validation_data=(np.stack(data_test['spectrogram']), np.stack(data_test['label'].values)))

When training

Epoch 50/50
466/466 [==============================] - 4s 8ms/step - loss: 0.5264 - acc: 0.7425 - val_loss: 0.8254 - val_acc: 0.5339

But when I do

train_preds = np.round(model.predict(np.stack(data_train['spectrogram']))).astype('int32')

The train_preds are around 55% accurate, however, as shown above, the reported accuracy on the last epoch is 0.7425

Am I doing something wrong? Or is there some explanation?

I'll be happy to post more of the code, if that helps.

Upvotes: 2

Views: 566

Answers (3)

Amir
Amir

Reputation: 16607

When training accuracy/loss is high but dev/test accuracy/loss is low, It clearly gives a signal of overfitting. If you plot your accuracy for training and testing data, you probably have something similar to the following picture:

enter image description here

At some point in your plot, training accuracy is decreased but test accuracy is not. What can be done?

  1. Early Stopping
  2. Regularization
  3. Train with more data
  4. Simplify your model (fewer parameters)

Clue: Based on your code, you have two LSTM and 466 training examples. LSTM model is relatively a powerful network and can easily overfit on the data. Decrease hidden layer size of LSTM or remove one of them may be a good starting point to avoid overfitting.

Upvotes: 0

LaSul
LaSul

Reputation: 2421

Your 74% is on the Training Set, this is an indicator of own is performing your algorithm while training, you should never look at it as a reference.

You should always look at your Test Set, this is the real value that matters.

Fore more, your accuracies should always look like this (at least the style):

e.g. The training set accuracy always growing and the testing set following the same trend but below the training curve.

enter image description here

Upvotes: 0

user9477964
user9477964

Reputation:

The accuracy shown in the epochs is the accuracy of the model on the training dataset you provided. And the model.predict shows the accuracy on the validation dataset. The 0.7 accuracy is on the training dataset and the 0.55 accuracy is on the test/validation dataset.

Upvotes: 1

Related Questions