CODEWITHSUNDEEP
pythonneural-networktheanoconv-neural-networknovelty-detection
Arsey
Arsey

Reputation: 281

Efficient way to know if an image related to a dataset that was used to train convolutional neural network

Currently I'm using VGG16 + Keras + Theano thought the Transfer Learning methodology to recognize plants classes. It works just fine and gives me a good accuracy. But the next problem I'm trying to solve - is to find a way of identifying if an input image contains plant at all. I don't want to have another one classifier that will do it, because it's not really efficiently.

So I did some search and have found that we can get activations from the latest model layer (before activation layer) and analyze it.

from keras import backend as K

model = util.load_model() # VGG16 model
model.load_weights(path_to_weights)

def get_activations(m, layer, X_batch):
    x = [m.layers[0].input, K.learning_phase()]
    y = [m.get_layer(layer).output]
    get_activations = K.function(x, y)
    activations = get_activations([X_batch, 0])

    # trying to get some features from activations
    # to understand how can we identify if an image is relevant
    for l in activations[0]:
        not_nulls = [x for x in l if x > 0]

        # shows percentage of activated neurons
        c1 = float(len(not_nulls)) / len(l)
        n_activated = len(not_nulls)
        print 'c1:{}, n_activated:{}'.format(c1, n_activated)

    return activations

get_activations(model, 'the_latest_layer_name', inputs)

From the above code I've noticed that when we have very irrelevant image, the number of activated neurons is bigger than for images that contain plants:

  1. For images that was using for model training, number of activated neurons 19%-23%
  2. For images that contain unknown plants species 20%-26%
  3. For irrelevant images 24%-28%

It's not really a good feature to understand if an image relevant as percentage values are intersect.

So, is there a good way to resolve this issue?

Upvotes: 3

Views: 302

Answers (1)

Arsey
Arsey

Reputation: 281

Thanks to Feras's idea in the comment above. After some trials, I've come up with the ultimate solution that allows solving this problem with accuracy up to 99.99%.

Steps are:

  1. Train your model on a dataset;
  2. Store activations (see method above how to get them) by predicting relevant and non-relevant images using trained model from the previous step. You should get activations from the penultimate layer. For VGG16 it's the last of two Dense(4096), for InceptionV3 - an extra penultimate Dense(1024) layer, for resnet50 - an extra penultimate Dense(2048) layer.
  3. Solve a binary problem using stored activations data. I've tried a simple flat NN and Logistic Regression. Both were good in accuracy (flat NN was a bit more accurate), but I've chosen the Logistic Regression as it's simpler, faster and consumes less memory and CPU/GPU.

This process should be repeated each time after your model retrained as each time the final weights for CNN are different and what was working previously, will be different next time.

So as result we have another small model for solving the problem.

Upvotes: 0

Related Questions