Keras: ValueError: Error when checking target: expected dense to have shape (10,) but got array with shape (400,)

Question

I am having an issue when trying to train my model in Keras, and a Tensorflow Backend.

import numpy as np
from sklearn.utils import shuffle

# Load Data
df = np.loadtxt("features.txt", delimiter=',') 
print('Features shape:', df.shape)

labels = np.loadtxt("labels.txt", delimiter=',') 
print('Labels shape', labels.shape)

# Replace 10 by 0
labels = np.where(labels == 10, 0, labels) 

# Randomize the data 
data_shuffled, labels_shuffled = shuffle(df, labels, random_state=42)

from keras.models import Sequential
from keras.layers import Dense

# split into input (X) and output (y) variables
X = data_shuffled[:4000]
y = data_shuffled[4000:]

print(X.shape, y.shape)

# define the keras model
model = Sequential()
model.add(Dense(12, input_dim=400, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(10, activation='sigmoid'))
# compile the keras model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit the keras model on the dataset
model.fit(X, y, epochs=1000, batch_size=100, verbose=0)
# make class predictions with the model
predictions = model.predict_classes(X)
# summarize the first 5 cases
for i in range(5):
    print('%s => %d (expected %d)' % (X[i].tolist(), predictions[i], y[i]))

# evaluate the keras model
accuracy = model.evaluate(X, y)
print('Accuracy: %.2f' % (accuracy*100))

Features shape: (5000, 400) Labels shape (5000,)

I got the error on the line where I call model.fit().

Things I have tried to fix this error: I tried to reshape the X and y numpy arrays using the following code: X.reshape(400, -1) y.reshape(400, -1)

But it doesn't help.

Answer 1

I think this line is wrong: y = data_shuffled[4000:]

Should be: y = labels_shuffled[:4000]

Then you need to one-hot-encode:

from sklearn.preprocessing import OneHotEncoder
onehotencoder = OneHotEncoder()
y = onehotencoder.fit_transform(y.reshape(-1,1)).toarray()