CODEWITHSUNDEEP
kerasclassificationconv-neural-networkpre-trained-model
Nicolas M.
Nicolas M.

Reputation: 1478

Keras - Classifier not learning from Transfer-Values of a Pre-Trained Model

I'm currently trying to use a pre-trained network and test in on this dataset. Originally, I used VGG19 and just fine-tuned only the classifier at the end to fit with my 120 classes. I let all layers trainable to maybe improve performance by having a deeper training. The problem is that the model is very slow (even if I let it run for a night, I only got couple of epochs and reach an accuracy of around 45% - I have a GPU GTX 1070).

Then, my thinking was to freeze all layers from this model as I have only 10k images and only train the few last Denses layers but it's still not realy fast.

After watching this video (at around 2 min 30s), I decided to replicate the principle of Transfer-Values with InceptionResnetv2.

I processed every pictures and saved the output in a numpy matrix with the following code. 

# Loading pre-trained Model + freeze layers
model = applications.inception_resnet_v2.InceptionResNetV2(
        include_top=False, 
        weights='imagenet', 
        pooling='avg')

for layer in model.layers:
    layer.trainable = False

# Extraction of features and saving
a = True
for filename in glob.glob('train/resized/*.jpg'):
    name_img = os.path.basename(filename)[:-4]
    class_ = label[label["id"] == name_img]["breed"].values[0]
    input_img = np.expand_dims(np.array(Image.open(filename)), 0)
    pred = model.predict(input_img)
    if a:
        X = np.array(pred)
        y = np.array(class_)
        a = False
    else:
        X = np.vstack((X, np.array(pred)))
        y = np.vstack((y, class_))

np.savez_compressed('preprocessed.npz', X=X, y=y)

X is a matrix of shape (10222, 1536) and y is (10222, 1).

After, I designed my classifier (several topologies) and I have no idea why it is not able to perform any learning.

# Just to One-Hot-Encode labels properly to (10222, 120)
label_binarizer = sklearn.preprocessing.LabelBinarizer()
y = label_binarizer.fit_transform(y)

model = Sequential()
model.add(Dense(512, input_dim=X.shape[1]))
# model.add(Dense(2048, activation="relu"))
# model.add(Dropout(0.5))
# model.add(Dense(256))
model.add(Dense(120, activation='softmax'))

model.compile(
    loss = "categorical_crossentropy", 
    optimizer = "Nadam", # I tried several ones
    metrics=["accuracy"]
)

model.fit(X, y, epochs=100, batch_size=64, 
          callbacks=[early_stop], verbose=1, 
          shuffle=True, validation_split=0.10)

Below you can find the output from the model :

Train on 9199 samples, validate on 1023 samples
Epoch 1/100
9199/9199 [==============================] - 2s 185us/step - loss: 15.9639 - acc: 0.0096 - val_loss: 15.8975 - val_acc: 0.0137
Epoch 2/100
9199/9199 [==============================] - 1s 100us/step - loss: 15.9639 - acc: 0.0096 - val_loss: 15.8975 - val_acc: 0.0137
Epoch 3/100
9199/9199 [==============================] - 1s 98us/step - loss: 15.9639 - acc: 0.0096 - val_loss: 15.8975 - val_acc: 0.0137
Epoch 4/100
9199/9199 [==============================] - 1s 96us/step - loss: 15.9639 - acc: 0.0096 - val_loss: 15.8975 - val_acc: 0.0137
Epoch 5/100
9199/9199 [==============================] - 1s 99us/step - loss: 15.9639 - acc: 0.0096 - val_loss: 15.8975 - val_acc: 0.0137
Epoch 6/100
9199/9199 [==============================] - 1s 96us/step - loss: 15.9639 - acc: 0.0096 - val_loss: 15.8975 - val_acc: 0.0137

I tried to change topologies, activation functions, add dropouts but nothing creates any improvements.

I have no idea what is wrong in my way of doing this. Is the X matrix incorrect ? Isn't it allowed to use the pre-trained model only as feature extractor then perform the classification with a second model ?

Many thanks for your feedbacks, Regards, Nicolas

Upvotes: 3

Views: 270

Answers (1)

Yu-Yang
Yu-Yang

Reputation: 14619

You'll need to call preprocess_input before feeding the image array to the model. It normalizes the values of input_img from [0, 255] into [-1, 1], which is the desired input range for InceptionResNetV2.

input_img = np.expand_dims(np.array(Image.open(filename)), 0)
input_img = applications.inception_resnet_v2.preprocess_input(input_img.astype('float32'))
pred = model.predict(input_img)

Upvotes: 2

Related Questions