What is the purpose of using genetic algorithm in learning an ANN

Question

I studied the basics of learning ANNs with a genetic algorithm. I found out that there are basically 2 things you can do:

1. Use GA to design the structure of the net (determine whether there should be an edge between two neurons or not). I guess we assume we can only use a certain amount of neuron-to-neuron connections.
2. Use GA to calculate optimal weights.

I also learned that GA makes sense only in case of irregular networks. If the net consists of layers it's suggested to use back propagation as it's faster.

If back propagation is faster and requires a network made of layers, why would I bother to choose GA for learning or designing the network?

Answer 1

Use GA to design the structure of the net (determine whether there should be an edge between two neurons or not).

In general you seem to be talking about feed forward networks, presumably MLPs.

With these structure of the network is concerned with the number of neurons and layers as well as the connections between neurons. Usually these are set out fully connected, so that every neuron in layer n is connected to every one in layer n+1. The training method will sort out partial connectivity by training some weights down to zero, or very small numbers.

There are some rules for setting up ANNs bases on the data complexity and what you want them to do. These can give you a good starting point. The training algorithms will sort out neuron to neuron connection but will not affect the number of neurons or layers.

So a GA can be used to experiment with parameters that affect the network size.

Use GA to calculate optimal weights.

GA's are not guaranteed to do this. "optimal weights" don't really exist. A trained network is going to give you a balance between recognition and error. You could say "optimal weights" to obtain a target error.

For a feed forward MLP a GA is going to take more processing time than Back Propagation.

I have also experienced that the GA does not fine tune as well so you may have a network that is less noise tolerant using a GA than an BP.

Neither approach is guaranteed to fine the absolute minimum or even an acceptable minimum. Both can get stuck in local minima. There are techniques for re-starting both GA and BP if this happens. But remember that your network's architecture may not allow it to achieve a an acceptable error on your data. There is a limited amount of memory/space in the weights and a solution my just not be there. So when you think you are in a local minima you could actually be at the absolute minim but above an acceptable error.

I also learned that GA makes sense only in case of irregular networks. If the net consists of layers it's suggested to use back propagation as it's faster.

You are right here, and not just with BP. Most network architectures that have a dedicated training algorithm are better suited to this than a GA.

But for irregular networks there may be no dedicated training algorithm. For these a GA allows you to experiment and train. Testing architectures to see if a solution is possible before you try to write a dedicated training algorithm.

Remember before BP was invented there was a ten year hiatus on ANNs because there was no method of training a MLP!

If back propagation is faster and requires a network made of layers, why would I bother to choose GA for learning or designing the network?

If you are using a FF network BP is usually the best LEARNING choice. However learning only involved manipulating the weights. A GA can be used to design the structure, and modify other things like Bias, squashing function,....

A point to note, which is often overlooked is that Back Propagation trains a single network by adjusting the weights. A GA is a population of many networks with fixed weights, it evolves a solution which is a network that is "born" with fixed weights. There is no actual network training/learning.

While the initial parameters of a single network; number of neurons, layers, bias, initial weights, may require attention and experimentation. The GA parameters of; population size, initial values, mutation rate, crossover,... all affect evolution time and the outcome of a likely or possible solution.

Answer 2

The structure of a network is not necessarily easy to choose (even for a layered one). The accuracy of the network will vary depending on how many neurons are used, how they are organized and connected and many other aspects. Using a GA algorithm to choose the setup might just produce better results than a human guess.

The same goes for the weights. Backpropagation does not necessarily produce a perfect result. It might only come up with a local optimum, which is performing worse than the network could with a different set of weights. A genetic algorithm might find better results here as well.

In the end, it is a different approach to solving the difficult optimization problems that ANNs pose.

Answer 3

Because you have no idea just how to organize your network into layers; in fact, you might use a GA in order to come up with a way to organize it into layers, and then use BP to calculate the weights in said network.