CODEWITHSUNDEEP
machine-learningkerasdeep-learning
Akash
Akash

Reputation: 113

validation accuracy is not increasing training ResNet50

I was doing fine-tuning with ResNet50 model for face recognition using data agumentation, but observed that the model accuracy was increasing but validation accuracy from the very starting point is not imporving, i am not getting where is it getting wrong, please review my code.

I have tried manipulating the top layers which i have added but it didn't helped. 

import os
os.environ['KERAS_BACKEND'] = 'tensorflow'
from keras.applications import ResNet50
from keras.models import Sequential
from keras.layers import Dense, Flatten, GlobalAveragePooling2D,Input,Dropout

num_classes = 13

base = ResNet50(include_top=False, weights='resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5',input_tensor=Input(shape=(100,100,3)))
from keras.models import Model

x = base.output

#x = GlobalAveragePooling2D()(x)

x = Flatten()(x)

x = Dense(1024, activation = 'relu')(x)

x = Dropout(0.5)(x)

predictions = Dense(13, activation='softmax')(x)

model = Model(inputs=base.input, outputs=predictions)

for layers in base.layers:
    layers.trainable= False

model.compile(optimizer='sgd', loss='categorical_crossentropy', metrics=['accuracy'])
from keras.preprocessing.image import ImageDataGenerator

train_generator = ImageDataGenerator(featurewise_center=True,
                                rotation_range=20,
                                rescale=1./255,
                                shear_range=0.2,
                                zoom_range=0.2,
                                width_shift_range=0.2,
                                height_shift_range=0.2,
                                horizontal_flip=True)

test_generator = ImageDataGenerator(rescale=1./255)

from sklearn.model_selection import train_test_split

x_train,x_test,y_train,y_test = train_test_split(image,label,test_size=0.2,shuffle=True,random_state=0)

train_generator.fit(x_train)
test_generator.fit(x_test)

model.fit_generator(train_generator.flow(x_train,y_train,batch_size=32),
                       steps_per_epoch =10,epochs=50, 
                    validation_data=test_generator.flow(x_test,y_test))

Output :

Epoch 19/50
10/10 [==============================] - 105s 10s/step - loss: 1.9387 - acc: 0.3803 - val_loss: 2.6820 - val_acc: 0.0709
Epoch 20/50
10/10 [==============================] - 107s 11s/step - loss: 2.0725 - acc: 0.3230 - val_loss: 2.6689 - val_acc: 0.0709
Epoch 21/50
10/10 [==============================] - 103s 10s/step - loss: 1.8884 - acc: 0.3375 - val_loss: 2.6677 - val_acc: 0.0709
Epoch 22/50
10/10 [==============================] - 95s 10s/step - loss: 1.8265 - acc: 0.4051 - val_loss: 2.6799 - val_acc: 0.0709
Epoch 23/50
10/10 [==============================] - 100s 10s/step - loss: 1.8346 - acc: 0.3812 - val_loss: 2.6929 - val_acc: 0.0709
Epoch 24/50
10/10 [==============================] - 102s 10s/step - loss: 1.9547 - acc: 0.3352 - val_loss: 2.6952 - val_acc: 0.0709
Epoch 25/50
10/10 [==============================] - 104s 10s/step - loss: 1.9472 - acc: 0.3281 - val_loss: 2.7168 - val_acc: 0.0709
Epoch 26/50
10/10 [==============================] - 103s 10s/step - loss: 1.8818 - acc: 0.4063 - val_loss: 2.7071 - val_acc: 0.0709
Epoch 27/50
10/10 [==============================] - 106s 11s/step - loss: 1.8053 - acc: 0.4000 - val_loss: 2.7059 - val_acc: 0.0709
Epoch 28/50
10/10 [==============================] - 104s 10s/step - loss: 1.9601 - acc: 0.3493 - val_loss: 2.7104 - val_acc: 0.0709

Upvotes: 4

Views: 4418

Answers (1)

Akash
Akash

Reputation: 113

It was happening because i was just directly adding the fully-connected layers without training it first, as mentioned in Blog of keras, https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

in order to perform fine-tuning, all layers should start with properly trained weights: for instance you should not slap a randomly initialized fully-connected network on top of a pre-trained convolutional base. This is because the large gradient updates triggered by the randomly initialized weights would wreck the learned weights in the convolutional base. In our case this is why we first train the top-level classifier, and only then start fine-tuning convolutional weights alongside it.

so answer is first train the top-model separately, then create a new model having ResNet50 model with its weight, with top-model and its weights on top of resnet model(base-model), then train it first by freezing the base-model(ResNet50) and the with the last layer of the base-model.

Upvotes: 3

Related Questions