CODEWITHSUNDEEP
pythonmachine-learningkerasdeep-learningclassification
anirudh
anirudh

Reputation: 423

Accuracy Decreasing with higher epochs

I am a newbie to Keras and machine learning in general. I'm trying to build a binary classification model using the Sequential model. After some experimenting, I saw that on multiple runs(not always) I was getting a an accuracy of even 97% on my validation data in the second or third epoch itself but this dramatically decreased to as much as 12%. What is the reason behind this ? How do I fine tune my model ? Here's my code - 

model = Sequential()
model.add(Flatten(input_shape=(6,size)))
model.add(Dense(6,activation='relu'))
model.add(Dropout(0.35))
model.add(Dense(3,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['binary_accuracy'])
model.fit(x, y,epochs=60,batch_size=40,validation_split=0.2)

Upvotes: 6

Views: 27444

Answers (3)

Shivaraj shavi
Shivaraj shavi

Reputation: 11

Reduce the learning rate (like 0.001 or 0.0001) and add batch normalization after every convolution layer.

Upvotes: 1

Padi Sarala
Padi Sarala

Reputation: 1

When I have this issue, I solved by changing the optimizer from RMSprop to adam. I also reduced the learning rate and added dropout after each fully connected layers. If your FC layers have small number of neurons then adding dropout does not make a major difference.

Upvotes: 0

user9477964
user9477964

Reputation:

According to me, you can take into consideration the following factors.

  1. Reduce your learning rate to a very small number like 0.001 or even 0.0001.
  2. Provide more data.
  3. Set Dropout rates to a number like 0.2. Keep them uniform across the network.
  4. Try decreasing the batch size.
  5. Using appropriate optimizer: You may need to experiment a bit on this. Use different optimizers on the same network, and select an optimizer which gives you the least loss.

If any of the above factors work for you, please let me know about it in the comments section.

Upvotes: 24

Related Questions