Keras 2d padding and input

Question

I am trying to feed a 2D CNN with images of different sizes. For that purpose (and since I do not wish to reshape my images) I try to perform a 2D padding.

The problem is the following : the images to be fed to the network are read and converted to 3-dimensionnal annoys (with shapes of the form : (80, 35, 3). Since the images are not of the same shape, I cannot create a np.array to put them in - so I append them in a list. Thus the famous error :

"Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 18418 arrays: " etc.

So the question is : how can I feed these images to the padding layer?

I have tried plenty of things like calling Input at the beginning :

    inputs = Input(shape = (None, None, 3,))

or configuring my padding layer as follows:

    model.add(ZeroPadding2D(padding=(64, 64), batch_input_shape= (1000, None, None, 3)))

but I can't get it right.

Would anyone have a solution?

Thanks in advance,

M

Answer 1

Padding

If you want to pad (which will make your model slow by performing too many unnecessary operations), you must do it outside of the model, with numpy.

If you have a list of images, like list of numpy arrays, each numpy with shape (side1,side2,3):

desiredX = someValue
desiredY = someValue
padded_images = []

for img in list_of_images:
    shape = img.shape
    xDiff = desiredX - shape[0]
    xLeft = xDiff//2
    xRight = xDiff-xLeft

    yDiff = desiredY - shape[1]
    yLeft = yDiff//2
    yRight = yDiff - yLeft

    padded_images.append(np.pad(img,((xLeft,xRight),(yLeft,yRight),(0,0)), mode='constant')
         #or choose another mode

padded_images = np.asarray(padded_images) #this can go into the model

Separate training

Alternatively you can train batches of one image, or small batches grouping images with the same size. (I don't know which will be more efficient, but if you have too big differences in sizes, this might be better)

for epoch in range(epochs):
    for img,output in zip(list_of_images,list_of_outputs):
        img = np.expand_dims(img,axis=0) #add the batch size = 1
        output = np.expand_dims(output,axis=0) ##add batch size = 1
        model.train_on_batch(img,output,....)