CODEWITHSUNDEEP
pythonmachine-learningneural-networkkeras
user9900027
user9900027

Reputation:

Input nodes in Keras NN

I am trying to create an neural network based on the iris dataset. I have an input of four dimensions. X = dataset[:,0:4].astype(float). Then, I create a neural network with four nodes.

model = Sequential()
model.add(Dense(4, input_dim=4, init='normal', activation='relu'))
model.add(Dense(3, init='normal', activation='sigmoid'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

As I understand, I pass each dimension to the separate node. Four dimensions - four nodes. When I create a neural network with 8 input nodes, how does it work? Performance still is the same as with 4 nodes.

model = Sequential()
model.add(Dense(8, input_dim=4, init='normal', activation='relu'))
model.add(Dense(3, init='normal', activation='sigmoid'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

Upvotes: 2

Views: 756

Answers (2)

Benjamin Kalish
Benjamin Kalish

Reputation: 36

To answer your main question of "How does this work?":

From a conceptual standpoint, you are initially creating a fully-connected, or Dense, neural network with 3 layers: an input layer with 4 nodes, a hidden layer with 4 nodes, and an output layer with 3 nodes. Each node in the input layer has a connection to every node in the hidden layer, and same with the hidden to the output layer.

In your second example, you just increased the number of nodes in the hidden layer from 4 to 8. A larger network can be good, as it can be trained to "look" for more things in your data. But too large of a layer and you may overfit; this means the network remembers too much of the training data, when it really just needs a general idea of the training data so it can still recognize slightly different data, which is your testing data.

The reason you may not have seen an increase in performance is likely either overfitting or your activation function; Try a function other than relu in your hidden layer. After trying a few different function combinations, if you don't see any improvement, you are likely overfitting.

Hope this helps.

Upvotes: 1

Ioannis Nasios
Ioannis Nasios

Reputation: 8527

You have an error on your last activation. Use softmax instead of sigmoid and run again.

replace

model.add(Dense(3, init='normal', activation='sigmoid'))

with

model.add(Dense(3, init='normal', activation='softmax'))

Upvotes: 2

Related Questions