Error when checking input: expected conv2d_1_input to have 4 dimensions, but got array with shape (800, 1000)

Question

i am trying to do sentimental analysis using CNN i my code my data has (1000,1000) shape when i pass the data to convolution2D it is throwing me an error. which i am not able to resolve. i tried below solution but still facing issue. When bulding a CNN, I am getting complaints from Keras that do not make sense to me.

My code is below.

TfIdf = TfidfVectorizer(max_features=1000) 
X = TfIdf.fit_transform(x.ravel()) 
Y = df.iloc[:,1:2].values


X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2,random_state=1)

classifier = Sequential()

classifier.add(Convolution2D(32, kernel_size=(3,3), input_shape=(1000, 1000, 1), activation = 'relu'))

classifier.add(MaxPooling2D(pool_size=(2,2)))

classifier.add(Flatten())

classifier.add(Dense(output_dim =  128, activation='relu'))

classifier.add(Dense(output_dim =  1, activation='sigmoid'))

classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

classifier.fit(X_train, Y_train, batch_size = 10, nb_epoch = 100, validation_data=(X_test,Y_test))

(loss,accuracy) = classifier.evaluate(X_test,Y_test, batch_size =10)

print(accuracy)

Answer 1

i added Embedding layer and replaced 2d convolutional layer with 1d. but my accuracy is constant even i changed the parameters.

here is my code.

classifier = Sequential()

classifier.add(Embedding(1000,64,input_length=1000))

classifier.add(Convolution1D(32, kernel_size=3, activation = 'relu'))

classifier.add(MaxPooling1D(pool_size=2))

classifier.add(Flatten())

classifier.add(Dense(output_dim =  128, activation='relu'))

classifier.add(Dense(output_dim =  1, activation='sigmoid'))

classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy']) 

Answer 2

Your neural network expects the data to be four-dimensional. Dimensions are (samples, rows, columns, channels). Your input data seems to be only two-dimensional. You need to add the first dimension which is samples since Keras expect to get more samples at the input. You can add a dimension for samples to your current input matrix with

X = X[np.newaxis, ...]

It will add the first dimension for samples which will have size 1. You also need to add the dimension for channels which is currently missing as the last dimension.

Both actions can be performed in one step with:

X = X[np.newaxis, ..., np.newaxis]

Answer 3

I may be wrong but for me you need to expand your data dimension in order to correspond to your network:

like:

X = np.expand_dims(X, axis=-1)