CODEWITHSUNDEEP
neural-network
mageta
mageta

Reputation: 687

Is there a rule/good advice on how big a artificial neural network should be?

My last lecture on ANN's was a while ago but I'm currently facing a project where I would want to use one.

So, the basics - like what type (a mutli-layer feedforward network), trained by an evolutionary algorithm (thats a given by the project), how many input-neurons (8) and how many ouput-neurons (7) - are set. But I'm currently trying to figure out how many hidden layers I should use and how many neurons in each of these layers (the ea doesn't modify the network itself, but only the weights).

Is there a general rule or maybe a guideline on how to figure this out?

Upvotes: 4

Views: 567

Answers (3)

Srikant Krishna
Srikant Krishna

Reputation: 881

The best approach for this problem is to implement the cascade correlation algorithm, in which hidden nodes are sequentially added as necessary to reduce the error rate of the network. This has been demonstrated to be very useful in practice.

An alternative, of course, is a brute-force test of various values. I don't think simple answers such as "10 or 20 is good" are meaningful because you are directly addressing the separability of the data in high-dimensional space by the basis function.

Upvotes: 3

corazza
corazza

Reputation: 32374

As rcarter suggests, trial and error might do fine, but there's another thing you could try.

You could use genetic algorithms in order to determine the number of hidden layers or and the number of neurons in them.

I did similar things with a bunch of random forests, to try and find the best number of trees, branches, and parameters given to each tree, etc.

Upvotes: 0

Romaine Carter
Romaine Carter

Reputation: 655

A typical neural net relies on hidden layers in order to converge on a particular problem solution. A hidden layer of about 10 neurons is standard for networks with few input and output neurons. However, a trial an error approach often works best. Since the neural net will be trained by a genetic algorithm the number of hidden neurons may not play a significant role especially in training since its the weights and biases on the neurons which would be modified by an algorithm like back propogation.

Upvotes: 2

Related Questions