CODEWITHSUNDEEP
pythonkerasvisualization
Abhijit Balaji
Abhijit Balaji

Reputation: 1940

Grad-cam not working properly in keras with inceptionv3

I am using visualise cam from keras-vis for creating guided-gradcam images. The grad-cam is working perfectly well with vgg16. but when i used the same code for inceptionv3 it is not working properly. 

    from keras.applications.inception_v3 import InceptionV3
    from vis.utils import utils
    from keras.preprocessing import image
    import numpy as np
    from keras import activations
    from matplotlib import pyplot as plt
    %matplotlib inline
    from vis.visualization import visualize_cam,overlay
    #build the inceptionv3 model with imagenet weights
    model = InceptionV3(weights='imagenet',include_top=True)

    # Utility to search for layer index by name
    layer_idx = utils.find_layer_idx(model,'predictions')

    #swap with softmax with linear classifier for the reasons mentioned above
    model.layers[layer_idx].activation = activations.linear
    model = utils.apply_modifications(model)
    from vis.utils import utils
    from matplotlib import pyplot as plt
    %matplotlib inline

    plt.rcParams['figure.figsize']=(18,6)

    img1 = utils.load_img('images/ouzel1.jpg',target_size=(299,299))
    img2 = utils.load_img('images/ouzel2.jpg',target_size=(299,299))
    f, ax = plt.subplots(1,2)
    ax[0].imshow(img1)
    ax[1].imshow(img2)
    plt.show()
    from vis.visualization import visualize_cam

    for modifier in [None, 'guided', 'relu']:
        plt.figure()
        f, ax = plt.subplots(1, 2)
        plt.suptitle("vanilla" if modifier is None else modifier)
        for i, img in enumerate([img1, img2]):    
            # 20 is the imagenet index corresponding to `ouzel`
            heatmap = visualize_cam(model, layer_idx, filter_indices=20, 
                                    seed_input=img, backprop_modifier=modifier, 
                                    #penultimate_layer_idx = 299 # corresponding to "conv2d_94"
                                    )
            # Lets overlay the heatmap onto original image.    
            ax[i].imshow(overlay(img, heatmap))

by commenting out the line #penultimate_layer also I am getting the same output which is not correct. can someone tell me what is the problem? The guided-grad cam result is given , followed by the original image is given.guided-gradcam image original image

The problem is the heatmap must be on the bird (ouzel).

Upvotes: 3

Views: 2264

Answers (1)

Corrado
Corrado

Reputation: 46

I hit the very same problem, but then I discovered that InceptionV3 mis-classifies these images. Check:

>>> model.predict(np.stack([img1, img2], 0)).argmax(axis=1) array([110, 725])

While with VGG it's:

>>> model.predict(np.stack([img1, img2], 0)).argmax(axis=1) array([20, 20])

Upvotes: 1

Related Questions