Deep Learning fit error (the list of Numpy arrays that you are passing to your model is not the size the model expected.)

Question

I'm new to Deep Learning. I'm trying to follow along with the fast.ai lecture series, and trying to reproduce the work manually in a Kaggle kernel.

I'm trying to work through cats vs. dogs Redux in Kaggle. I'm not concerned with accuracy, I just want to get something working.

I'm using Keras and the VGG16 model, as outlined in the fast.ai course. I'm also leaning on code outlined in this article to get me off the ground.

This is my Kaggle notebook.

I'm encountering an error when trying to fit my model that I don't know how to interpret:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-26-596f25281fc2> in <module>()
     12 #model.fit(input[0].transpose(), output[0].transpose())
     13 
---> 14 model.fit(X, Y, epochs=100, batch_size=6000, verbose=1)

/opt/conda/lib/python3.6/site-packages/Keras-2.1.2-py3.6.egg/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
   1591             class_weight=class_weight,
   1592             check_batch_axis=False,
-> 1593             batch_size=batch_size)
   1594         # Prepare validation data.
   1595         do_validation = False

/opt/conda/lib/python3.6/site-packages/Keras-2.1.2-py3.6.egg/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_batch_axis, batch_size)
   1428                                     output_shapes,
   1429                                     check_batch_axis=False,
-> 1430                                     exception_prefix='target')
   1431         sample_weights = _standardize_sample_weights(sample_weight,
   1432                                                      self._feed_output_names)

/opt/conda/lib/python3.6/site-packages/Keras-2.1.2-py3.6.egg/keras/engine/training.py in _standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
     81                 'Expected to see ' + str(len(names)) + ' array(s), '
     82                 'but instead got the following list of ' +
---> 83                 str(len(data)) + ' arrays: ' + str(data)[:200] + '...')
     84         elif len(names) > 1:
     85             raise ValueError(

ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 24500 arrays: [array([[1],
       [0]]), array([[1],
       [0]]), array([[0],
       [1]]), array([[1],
       [0]]), array([[1],
       [0]]), array([[1],
       [0]]), array([[1],
       [0]]), array([[0],
     ...

Here's some more information:

X = np.array([i[0] for i in train]).reshape(-1,IMG_SIZE,IMG_SIZE,3)
Y = [i[1] for i in train]

> type(X)
numpy.ndarray

> X.shape
(24500, 50, 50, 3)

> type(Y)
list

> len(Y)
24500

> Y[0]
[1 0]

> model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_7 (InputLayer)         (None, 50, 50, 3)         0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 50, 50, 64)        1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 50, 50, 64)        36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 25, 25, 64)        0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 25, 25, 128)       73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 25, 25, 128)       147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 12, 12, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 12, 12, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 12, 12, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 12, 12, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 6, 6, 256)         0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 6, 6, 512)         1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 6, 6, 512)         2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 6, 6, 512)         2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 3, 3, 512)         0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 3, 3, 512)         2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 3, 3, 512)         2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 3, 3, 512)         2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 1, 1, 512)         0         
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________

And the model:

model = VGG16(weights='imagenet', include_top=False, input_shape=(img_rows, img_cols, img_channel))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X, Y, epochs=100, batch_size=6000, verbose=1)

I've googled around but I'm at a loss for how to interpret this. This SO question seems similar, and seems to indicate that the output is the problem, but I'm not sure how that would apply to me here.

Answer 1

You should simply transform your Y to a numpy array with shape (24500, 2):

Y = np.ndarray(Y)