CODEWITHSUNDEEP
normalizationneural-networkgenetic-algorithm
Alek Andreev
Alek Andreev

Reputation: 21

How to normalize neural networks for genetic algorithms?

I am trying to train a simple feedforward neural network using a genetic algorithm, however it is proving fairly inefficient because isomorphic neural networks appear different to the genetic algorithm.

It is possible to have multiple neural networks, which behave the same way, but have their neurons ordered in a different way from left to right and across levels. To the genetic algorithms those networks' genotypes will appear completely different. Therefore any attempt to do crossover is pointless and the GA ends up being as effective as hill climbing.

Can you recommend a way to normalize the networks so they appear more transparent to the genetic algorithm?

Upvotes: 2

Views: 732

Answers (1)

Predictor
Predictor

Reputation: 994

I would call crossover in this context "inefficient", rather than "pointless". One way to address the duplication you mention might be to sort the hidden layer neurons in some canonical order, and use this order during crossover, which might at least reduce the duplication encountered in hidden weight space.

Also, you might fit the output layer weights by a more direct method than genetic algorithms. You don't say what performance metric is being used, but many common measures have fairly straightforward optimizations. So, as an example, you might generate a new hidden layer using genetic operators, then fit the output layer by logistic regression, and have the GA evaluate the total network.

Upvotes: 0

Related Questions