CODEWITHSUNDEEP
pythonkerasconv-neural-network
user1850484
user1850484

Reputation: 388

Prediction by Neural network Keras is not working

I want to write a NN program for recognition using Keras.

I have used 2 sets of data for training:

toyX = [1, 2, 3, 4, 5, 6, 7, 8]
toyX2 = [18, 17, 16, 15, 14, 13, 12, 11].

After training with toyX and then toyX2, the output of model.predict(toyX) is [[0.56053144 1.0758346 1.7890009 ]]. However, it should have been [6, 11, 14].

Should I add more layers or change parameters to improve prediction? Which parameters I should change?

Please help me to solve this problem.

from keras.models import Sequential
from keras.optimizers import Adam
from keras.layers import Conv1D, MaxPooling1D
from keras.layers import Dense, Flatten
from keras.layers import Dropout
import numpy as np

model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(8, 1)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
#model.add(Dense(3, activation='softmax'))
model.add(Dense(3))
#model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
print(model.summary())


toyX = np.array([1, 2, 3, 4, 5, 6, 7, 8]).reshape(1, 8, 1)
toyX2 = np.array([18, 17, 16, 15, 14, 13, 12, 11]).reshape(1, 8, 1)
#print (toyX.shape)
toyY = np.array([6, 11, 14]).reshape(1, 3)
#print (toyY.shape)
toyY2 = np.array([1, 2, 3]).reshape(1, 3) # if flatten is active

model.fit(toyX, toyY, epochs = 1000, verbose = 0)
model.fit(toyX2, toyY2, epochs = 1000, verbose = 0)

print (model.predict(toyX))

Upvotes: 0

Views: 64

Answers (1)

Zabir Al Nazi Nabil
Zabir Al Nazi Nabil

Reputation: 11228

This phenomenon is called catastrophic forgetting of neural networks.

You can read a paper on this: https://arxiv.org/pdf/1708.02072.pdf

Your toyX and toyX2 has a completely different distribution. When you re-train your model with ToyX2 for 1000 epochs, your model has completely forgotten the mapping from toyX to toyY.

You should train only for very few epochs with a really small learning rate if you want to make sure the knowledge of previous training stays or just mix both of the sets and re-train.

Upvotes: 1

Related Questions