CODEWITHSUNDEEP
pythonkerasconv-neural-network
DenseLayerIsWhat
DenseLayerIsWhat

Reputation: 21

What should the second to last dense layer be?

When looking online at Keras CNN examples, I always see the last two layers being fully connected layers like..

model.add(Flatten()) 
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(3))

I understand the last Dense layer and the fact that the number is 3 because in this specific case, that is the amount of possible classes to choose from.

What I don't understand is why the second to last Dense layer is 64. Sometimes it's 64, or 128 or 512.

My question is what does the 64 mean in model.add(Dense(64)) and what determines this number?

Upvotes: 1

Views: 1768

Answers (1)

malioboro
malioboro

Reputation: 3281

64 is the number of neuron in that layer. Why 64? there are no "exact" rules for number of neuron in hidden layer. May be it is a result of tuning parameter, or it chosen because their machine can't run with 100 neuron, or something else.

When you create a neural network, number of neuron in a layer is one of many hyper-parameters you should tune to find the best result. And also the number of hidden layer (don't you ask why there are 1 fully connected hidden layers?) or activation function.

I've read from https://stats.stackexchange.com/a/1097/105981: 'the optimal size of the hidden layer is usually between the size of the input and size of the output layers', but in many cases that I've ever done, that's not happen

Upvotes: 3

Related Questions