CODEWITHSUNDEEP
pythonkerasvgg-net
stackersTech101
stackersTech101

Reputation: 107

Using Keras VGG19 preprocess_input function during model training

I am using VGG19 model pretrained on ImageNet dataset with top-layers (from flatten till the last output layer) removed.

The problem I am solving takes in multiple inputs that are from ModelNet dataset (a modified version actually). So what I am doing is, I am using VGG19 to extract features from these images for me and then I concatenate the output which is further fed to the rest of the network i.e. the top of the model that I have created according to my need and the number of classes (20 in my case). This is my code for the model I am using (if needed for reference):

from keras.applications.vgg19 import VGG19

input_1 = Input(shape=(224, 224, 3), name='image1')              
input_2 = Input(shape=(224, 224, 3), name='image2')              
input_3 = Input(shape=(224, 224, 3), name='image3')              
input_4 = Input(shape=(224, 224, 3), name='image4')              

base_model = VGG19(weights='imagenet', input_shape=(224, 224, 3), include_top=False)
base_model.trainable = False

x1 = base_model(input_1, training=False)
x2 = base_model(input_2, training=False)
x3 = base_model(input_3, training=False)
x4 = base_model(input_4, training=False)

x = concatenate([x1, x2, x3, x4])

x = Flatten()(x)
x = Dense(512, activation='relu')(x)
x = Dropout(0.5)(x) 
x = Dense(256, activation='relu')(x)
outputs = Dense(20, activation='softmax', name='class_out')(x)

model = tf.keras.models.Model([input_1, input_2, input_3, input_4], outputs)
print(model.summary())

Now, I want to know that is it necessary to use 'from keras.applications.vgg19 import preprocess_input' for my input images before training the model? or should I use this function while predicting only? As for my input, they are already normalized values that I am using for model training via a custom made data-loader function; which simply returns the generator of 4 normalized input images and a class_output, like this:

# The output from data-loader function
# where x_batch are values normalized by x_batch[i,j]/255
 yield {'image1': x_batch[:, 0], 
          'image2': x_batch[:, 1], 
          'image3': x_batch[:, 2], 
          'image4': x_batch[:, 3], }, {'class_out': y_batch}

However, whenever I use preprocess_input instead of plain normalization, my output image is a weird looking image like this: processed image with keras preprocess_input

I do not understand this scenario.. if anyone can help me with this please.

Upvotes: 0

Views: 1681

Answers (1)

MD Mushfirat Mohaimin
MD Mushfirat Mohaimin

Reputation: 2066

VGG19 requires your input image to be a BGR image, not RGB image. And so, keras.applications.vgg19.preprocess_input will convert the input images from RGB to BGR, then will zero-center each color channel with respect to the ImageNet dataset, without scaling. For more information about your model click here, and more about the preprocessing function can be found here

So, if you are already converting your images from RGB to BGR before passing to the model, then you don't need to use keras.applications.vgg19.preprocess_input on your input images.

Upvotes: 1

Related Questions