Different Input/Output shape in Keras

Question

I'm new to all this Neural Networks thing and I'm actually trying some toy codes with different codig options (raw Python, TF...)

Currently, I've made a simple binary AND, OR and NOT operator solving network in TFLearn:

# 1. Import library of functions
import numpy as np
import tflearn
from keras.models import Sequential
from keras.layers import Dense, Activation

# 2. Logical data
input = [[0., 0.], [0., 1.], [1., 0.], [1., 1.]]
YOR = [[0.], [1.], [1.], [1.]]
YAND=[[0.], [0.], [0.], [1.]]
YNOT=[[0.], [1.], [1.], [0.]]

######   VERSION TFLEARN     #####
# 3. Building our neural network/layers of functions 
neural_net = tflearn.input_data(shape=[None, 2])
neural_net = tflearn.fully_connected(neural_net, 1, activation='sigmoid')
neural_net = tflearn.regression(neural_net, optimizer='sgd', learning_rate=2, loss='mean_square')

# 4. Train the neural network / Epochs
model = tflearn.DNN(neural_net,tensorboard_verbose=0)
model.fit(input, YOR, n_epoch=1000, snapshot_epoch=False)

# 5. Testing final prediction
print("Testing OR operator")
print("0 or 0:", model.predict([[0., 0.]]))
print("0 or 1:", model.predict([[0., 1.]]))
print("1 or 0:", model.predict([[1., 0.]]))
print("1 or 1:", model.predict([[1., 1.]]))

Now I'm trying to replicate it in Keras (using CNTK backend) using this code:

# 2. Logical OR operator / the data
input = np.array([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])
YOR = np.array([[0.], [1.], [1.], [1.]])
YAND=np.array([[0.], [0.], [0.], [1.]])
YNOT=np.array([[0.], [1.], [1.], [0.]])

######   VERSION KERAS     #####
# 3. Building our neural network/layers of functions 
model= Sequential()
model.add(Dense(4,input_shape=[2,]))
model.add(Activation('sigmoid'))

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# 4. Train the neural network / Epochs
model.fit(input,YOR,epochs=1000,verbose=1)

# 5. Testing final prediction
print("Testing OR operator")
print("0 or 0:", model.predict([[0., 0.]]))
print("0 or 1:", model.predict([[0., 1.]]))
print("1 or 0:", model.predict([[1., 0.]]))
print("1 or 1:", model.predict([[1., 1.]]))

On execution, I would expect to obtain the result of the operator in each case, but instead I got the following error:

ValueError: Error when checking input: expected dense_1_input to have shape (2,) but got array with shape (1,)

According to Keras Doc, seems to be that the output shape must be the same as the input shape, and though I can modify the input_shape, apparently doesn't recognize the output_shape arg.

By the way if I try to change the value of the input_shape in order to fit it to the output (according to what i just mention) I get the same message but swapping those values.

Does this mean that I can only obtain results of the same shape as the input?

Answer 1

I want to add something to the already given answer. Because you actually can keep the line as it was with 4 units resp. hidden size:

model.add(Dense(4, input_shape=(2,))) 

So assuming you want to keep a hidden size of 4, then you need to add just an proper output layer where the shape is matching the shape of your data.

In you case:

model.add(Dense(1))

So if you want to keep the hidden size different than 1 this is probably what you want, here is the full working code:

Note: I also added another activation for the output layer.

import numpy as np

from keras.models import Sequential
from keras.layers import Dense, Activation

# 2. Logical OR operator / the data
input = np.array([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])
YOR = np.array([[0.], [1.], [1.], [1.]])
YAND=np.array([[0.], [0.], [0.], [1.]])
YNOT=np.array([[0.], [1.], [1.], [0.]])

######   VERSION KERAS     #####
# 3. Building our neural network/layers of functions 
model= Sequential()
model.add(Dense(4, input_shape=(2,)))
# you can place 
model.add(Activation('sigmoid'))
# layer to match output shape
model.add(Dense(1))
# of course you can add a sigmoid or other 
# activation here to match you target value range
model.add(Activation('sigmoid'))

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# 4. Train the neural network / Epochs
print(input.shape)
model.fit(input,YOR,epochs=1000,verbose=1)

# 5. Testing final prediction
print("Testing OR operator")
print("0 or 0:", model.predict([[0., 0.]]))
print("0 or 1:", model.predict([[0., 1.]]))
print("1 or 0:", model.predict([[1., 0.]]))
print("1 or 1:", model.predict([[1., 1.]]))

I hope this makes things clearer and helps you to understand the error message better.

Answer 2

I tried running the program you have given. But it produced a different type of error for me

Error when checking target: expected activation_13 to have shape (4,) but got array with shape (1,)

I changed value inside Dense to solve the above error. Why don't you try using this

model= Sequential()
model.add(Dense(1,input_shape=(2,)))
model.add(Activation('sigmoid'))

model.compile(optimizer='rmsprop',
          loss='binary_crossentropy',
          metrics=['accuracy'])

# 4. Train the neural network / Epochs
model.fit(input,YOR,epochs=1000,verbose=1)

# 5. Testing final prediction
print("Testing OR operator")
test = np.array([[0., 0.]])
print("0 or 0:", model.predict(test))
test = np.array([[0., 1.]])
print("0 or 1:", model. model.predict(test))
test = np.array([[1., 0.]])
print("1 or 0:",  model.predict(test))
test = np.array([[1., 1.]])
 print("1 or 1:",  model.predict(test))

Also we can train models in Keras even if the input and output shape are different