CODEWITHSUNDEEP
pythontensorflowkerasdeep-learning
Darkstar Dream
Darkstar Dream

Reputation: 1859

How to solve constant model accuracy after each epoch

I am studying deep learning and as an assignment, I am doing a classification project, which has 17k records with 14 features and a target variable that have 11 classes. enter image description here

I tried to train a simple neural network

# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='relu'))
model1.add(keras.layers.Dense(128, activation='relu'))
model1.add(keras.layers.Dense(64, activation='relu'))  
model1.add(keras.layers.Dense(1, activation='softmax'))

# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

# fit the keras model on the dataset
performance1 = model1.fit(x_train, y_train, epochs=100, validation_split=0.2)

But the problem here is I am getting the same accuracy for each epoch, it doesn't seem that model is even learning. enter image description here

I tried to research this problem and found some similar problems on StackOverflow like this question and tried following things

But none of them worked, of course, the accuracy was different in the different trials (but has the same problem).

Few of trials are as follows:

# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='sigmoid'))
model1.add(keras.layers.Dense(128, activation='sigmoid'))
model1.add(keras.layers.Dense(64, activation='sigmoid'))  
model1.add(keras.layers.Dense(1, activation='softmax'))

sgd = keras.optimizers.SGD(lr=0.01)
# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='relu'))
model1.add(keras.layers.Dense(128, activation='relu'))
model1.add(keras.layers.Dropout(0.2))
model1.add(keras.layers.Dense(64, activation='relu')) 
model1.add(keras.layers.Dropout(0.2))
model1.add(keras.layers.Dense(1, activation='softmax'))

# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

I don't know what's the problem here. Please let me know if you require more details to process this. And please don't close this question I know this question stands a chance to marked as a duplicate question but trust me I tried many things which I can understand as a beginner.

Upvotes: 0

Views: 825

Answers (1)

Abhishek Prajapat
Abhishek Prajapat

Reputation: 1878

The problem is that the softmax should be applied on an output array to get probabilities and that output array from the model should represent the logits for each target class. hence you would have to change this line

model1.add(keras.layers.Dense(1, activation='softmax'))
# TO
model1.add(keras.layers.Dense(df['Class'].nunique(), activation='softmax'))

EDIT:

# Let's say you have 11 unique values in your class then you last layer will become
model1.add(keras.layers.Dense(11, activation='softmax'))

# Now your loss will be
model1.compile(loss=tf.keras.loss.SparseCategoricalCrossentropy(), optimizer='adam', metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])

Upvotes: 1

Related Questions