CODEWITHSUNDEEP
pythontensorflowkerasdeep-learningconv-neural-network
Sree
Sree

Reputation: 983

Keras VGG16 modified model giving the same prediction every time

I have taken the inbuilt keras.applications.vgg16.VGG16(weights='imagenet', include_top=True,input_shape=(224,224,3)) model and did transfer learning for PASCAL VOC 2012 dataset with 20 classes by including Global Average Pooling Layer as given below:

def VGG16_modified():
    base_model = vgg16.VGG16(include_top=True,weights='imagenet',input_shape=(224,224,3))
    print(base_model.summary())
    x = base_model.get_layer('block5_pool').output
    x = (GlobalAveragePooling2D())(x)
    predictions = Dense(20,activation='sigmoid')(x)

    final_model = Model(input = base_model.input, output = predictions)
    print(final_model.get_weights())
    return final_model

Now, I want to take the Class Activation Map based on this paper. For this, my code is as given below:

 def get_CAM(model,img):
        model = load_model(model)
        im = image.load_img(img,target_size=(224,224))
        im = image.img_to_array(im)
        im = np.expand_dims(im,axis=0)
        class_weights = model.layers[-1].get_weights()[0]
        final_conv_layer = model.get_layer('block5_pool')
        cam_model = Model(inputs = model.input,outputs=(final_conv_layer.output,model.layers[-1].output))
        conv_outputs, predictions = cam_model.predict(im)
        conv_outputs = np.squeeze(conv_outputs)
        prediction = np.argmax(predictions)
        print(predictions)
        print(prediction)
        print(conv_outputs)
        print(conv_outputs.shape)
        class_weights = class_weights[:,prediction]
        mat_for_mult = scipy.ndimage.zoom(conv_outputs,(32,32,1),order=1)
        final_output = np.dot(mat_for_mult.reshape((224*224, 512)),class_weights).reshape((224,224))
        print(final_output)
        return final_output

But cam_model.predict(im) is always giving the same class for all images. I am not sure where have I wrong with this. As the pascal voc 2012 contains multi label images, I have used 'sigmoid' in the last layer of the modified_vgg16 rather than 'softmax'. Can you let me know where have I gone wrong.

Upvotes: 0

Views: 788

Answers (1)

karandeep36
karandeep36

Reputation: 336

PASCAL VOC has imbalanced distribution of images between classses.

Most of the images in dataset are for person which leads to imbalance data distribution and creates bias towards person class.

In order to avoid it, you can use the method of weighting the classes which helps to remove imbalance between classes

Please refer to this for more information on how to set class weights for imbalanced class.

Upvotes: 1

Related Questions