CODEWITHSUNDEEP
pythonmachine-learningkerasneural-network
Joshua Potts
Joshua Potts

Reputation: 13

Simple keras neural network stuck at 87.5% accuracy

I'm learning the Keras library and have come into a problem. I'm creating a very simple neural network. It takes an input of three digits (eg. 010) and outputs a single digit (eg.1). It should only output 1 if there is a 1 in the input three digits.

However, when I run my code, the accuracy stays at 87.5% for the full five epochs. This tells me it is simply not training for the 000 instance. Why is it not changing? I don't see what I did wrong :(

from keras.datasets import mnist
from keras import models
from keras import layers
from keras.utils import to_categorical
import time
import numpy as np

train_images = np.array([[0,0,0],[0,0,1],[0,1,0],[0,1,1],[1,0,0],[1,0,1],[1,1,0],[1,1,1]])
train_labels = np.array([[0],[1],[1],[1],[1],[1],[1],[1]])


train_images = train_images.reshape((8, 3))


model = models.Sequential()

model.add(layers.Dense(6, input_dim=3, activation='relu'))
model.add(layers.Dense(6, activation='relu'))
model.add(layers.Dense(1, activation='softmax'))




model.compile(optimizer='adam',
                loss='categorical_crossentropy',
                metrics=['accuracy'])

model.fit(train_images, train_labels, epochs=5, batch_size=4)

Upvotes: 1

Views: 138

Answers (1)

Celius Stingher
Celius Stingher

Reputation: 18367

The model is stuck at 87.5% because it is always predicting class=1 as soon as from the first epoch. Given you only have 8 data points, a simple epoch is enough for the network to learn as much as it can, that's why you see no progress between the first and final epoch.

You have 8 data points, 7 of them are class 1 and 1 is class 0. Your model predicting always class = 1 means the accuracy is 7/8 = 87.5%

Regarding the model itself (its architecture), I believe it has 49 parameters which is too much in my opinion and as I previously mentioned, enough for it as much as it can in a single epoch. Also, if you are facing a binary classification problem, you might want to:

  • Change softmax for sigmoid
  • Change categorical crossentropy for binary crossentropy.

Upvotes: 1

Related Questions