CODEWITHSUNDEEP
neural-network
Androme
Androme

Reputation: 2449

Artificial Neural Networks: Choosing initial neurons

How is the initial structure (Neurons and connections between them) chosen? My book only states that we give the connection random weights in the beginning before we train the network.

I think that we would add neurons during the training like this:

  1. Start with a completely empty network
  2. The first value I generate during the training will not exist
  3. Add a neuron to correspond to this value, with a random weight

Upvotes: 2

Views: 245

Answers (2)

Peace Makes Plenty
Peace Makes Plenty

Reputation: 866

I do not believe the question is about self-organising or GA-evolved ANNs. It sounds more like it is about a the most common ANN: a perceptron (single or multi-layer), in which case the structure of the network: the number of layers and the size of the layers, must be hand chosen at the beginning. A simple initial rule of thumb for initialising the weight is simply picking uniformly random values between -1.0 and 1.0.

Upvotes: 0

Dervall
Dervall

Reputation: 5744

What you are after is a self-organizing ANN. Usually, the way the connections are organized is man-made into a model that the developer thinks will have sufficient power to perform the computation neccessary. You can of course start with a random selection of nodes with random connections, but the evolution of such a network will probably take a lot longer time than a standard two or three layer network.

So, yes, you are right in that you would use a similar approach when doing a self-organizing network. Keep track of two sets of genetic algorithms, one for the structure and one for the weights (or combine the two in some devious way) and evolve as you please.

Upvotes: 2

Related Questions