CODEWITHSUNDEEP
machine-learningkerasneural-networkdeep-learningconv-neural-network
Nevin Baiju
Nevin Baiju

Reputation: 344

Implementation of AlexNet in Keras on cifar-10 gives poor accuracy

I tried implementing AlexNet as explained in this video. Pardon me if I have implemented it wrong, this is the code for my implementation it in keras.

Edit : The cifar-10 ImageDataGenerator

cifar_generator = ImageDataGenerator()

cifar_data = cifar_generator.flow_from_directory('datasets/cifar-10/train', 
                                                 batch_size=32, 
                                                 target_size=input_size, 
                                                 class_mode='categorical')

The Model described in Keras:

model = Sequential()

model.add(Convolution2D(filters=96, kernel_size=(11, 11), input_shape=(227, 227, 3), strides=4, activation='relu'))
model.add(MaxPool2D(pool_size=(3 ,3), strides=2))

model.add(Convolution2D(filters=256, kernel_size=(5, 5), strides=1, padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(3 ,3), strides=2))

model.add(Convolution2D(filters=384, kernel_size=(3, 3), strides=1, padding='same', activation='relu'))
model.add(Convolution2D(filters=384, kernel_size=(3, 3), strides=1, padding='same', activation='relu'))
model.add(Convolution2D(filters=256, kernel_size=(3, 3), strides=1, padding='same', activation='relu'))

model.add(MaxPool2D(pool_size=(3 ,3), strides=2))

model.add(Flatten())
model.add(Dense(units=4096))
model.add(Dense(units=4096))
model.add(Dense(units=10, activation='softmax'))

model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])

I have used an ImageDataGenerator to train this network on the cifar-10 data set. However, I am only able to get an accuracy of about .20. I cannot figure out what I am doing wrong.

Upvotes: 0

Views: 2504

Answers (2)

mak_97
mak_97

Reputation: 1

i think you should scale down the input shape from 227,227,3 to 32,32,3(because of cifar images shape). Also try reducing the size of kernels from (11,11) to (3,3) and stride to 1. y doing so you are preventing the details from getting missed out.Also you need to reduce th neurons of fc layers from 4096 to 1028.These are the basic steps you can do to get the model work. Even then there are a lot of steps involved to get your model reach decent accuracy.

Upvotes: 0

desertnaut
desertnaut

Reputation: 60400

For starters, you need to extend the relu activation to your two intermediate dense layers, too; as they are now:

model.add(Dense(units=4096))
model.add(Dense(units=4096))

i.e. with linear activation (default), it can be shown that they are equivalent to a simple linear unit each (Andrew Ng devotes a whole lecture in his first course on the DL specialization explaining this). Change them to:

model.add(Dense(units=4096, activation='relu'))
model.add(Dense(units=4096, activation='relu'))

Check the SO thread Why must a nonlinear activation function be used in a backpropagation neural network?, as well as the AlexNet implementations here and here to confirm this.

Upvotes: 1

Related Questions