CODEWITHSUNDEEP
pythontensorflowkeraslstm
Erland Devona
Erland Devona

Reputation: 193

Keras LSTM XOR gate

I was learning about Keras for a project, for trial I try to make a simple machine learning using LSTM (what I will use for the project) for simple XOR gate prediction. But, even though I change number of neuron, layer, loss function, epoch, or optimizer I can't get the correct prediction. Is there something I miss about Keras or this code?

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, LSTM

data = [[[0, 0]], [[0, 1]], [[1, 0]], [[1, 1]]]
output = [[1, 0], [0, 1], [0, 1], [1,0]]

model = Sequential()
model.add(LSTM(10, input_shape=(1, 2), return_sequences=True))
model.add(LSTM(10))
model.add(Dense(2))

model.compile(loss='mae', optimizer='adam')
# print(model.summary())
model.fit(np.asarray(data), np.asarray(output), epochs=50)

print(model.predict_classes(np.asarray(data)))

Upvotes: 1

Views: 996

Answers (1)

nuric
nuric

Reputation: 11225

You are predicting XOR output encoded as one-hot vectors. In this case, it is much like a classification problem. If you use softmax to produce a distribution and set your loss to categorical_crossentropy your network starts to learn:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, LSTM

data = [[[0, 0]], [[0, 1]], [[1, 0]], [[1, 1]]]
output = [[1, 0], [0, 1], [0, 1], [1,0]]

model = Sequential()
model.add(LSTM(10, input_shape=(1, 2), return_sequences=True))
model.add(LSTM(10))
model.add(Dense(2, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer='adam')
# print(model.summary())
model.fit(np.asarray(data), np.asarray(output), epochs=200)

print(model.predict_classes(np.asarray(data)))

Also you would need to increase the number of epochs as adam default values have small learning rate.

Upvotes: 2

Related Questions