CODEWITHSUNDEEP
artificial-intelligenceneural-networkgenetic-algorithmnumerical-methodsevolutionary-algorithm
Troglodit
Troglodit

Reputation: 11

Neural Network training method

I've been studying Neural Networks lately. I'll explain my goal: i'm trying to teach monsters to walk, stand, basically perform actions that "reward" them (maximize the fitness function). The NN receives sensor inputs, and outputs muscle activity. The problem gets down to training the weights and biases of the neurons. My problem is that i'm not sure if i'm doing things right, and with neural networks i can make a mistake and never know about it. So i'll explain what i'm doing in general, and if you spot a mistake please correct me!

1) I create a neural network with neurons that use hyperbolic tangent transfer function.

2) Create a population of random "Chromosomes", each containing an array of doubles as genes(the weights and biases in the NN), the length of the array being amount of weights and biases in the NN. The genes have a lower and upper limit, usually [-2,2] in which their random value is generated in initialization and mutation.

For each generation:

3) For each chromosome, I update the NN weights and test the monster for about 5000 frames. Every 10 frames, network outputs are generated with sensor input. The outputs are double values normalized to [0,1] and they control "muscles" (springs) in the body by changing their neutral length, according to that value. Fitness value is calculated.

4) Perform Genetic Algorithm operators- first create cross overs with ~0.4 probability, then mutate with ~0.1 probability, depending on chromosome length. Mutation randomizes the gene to a value between some lower and upper limit. Elitism - two best solutions are left unchanged for the next generation.

Repeat until generations>maxGenerations or max fitness is reached.

I'm not sure about a few things in my code: should there be a limit for weights and biases? if yes, it constricts the potential results the NN could achieve. If no, then how do i initialize values, and mutate? I'm afraid that adding a random value as mutation will get stuck in local optima, like hill climbing. No limit will reduce the amount of parameters i need to consider when initializing the whole thing, which is nice!

Is hyperbolic tangent a good choice? why or why not?

Do i have to normalize inputs sensor data? if yes, between what values?

Also i'm not sure if i'm doing a mistake by outputting a double value for flexing instead of binary- higher than 0.5 is flex, less is release, could be an option, when now i'm just using the value as flex amount.

Don't consider bugs in my code as reasons for bad results, because i checked many times and implemented XOR that worked perfectly.

I would greatly appreciate any help, thank you!

Upvotes: 0

Views: 214

Answers (1)

jcfgonc
jcfgonc

Reputation: 31

I assume you are referring to Feed Forward Neural Networks, ie, forward connected layers of neurons.

It's ok to use hyperbolic tangent or a sigmoid function. Just make sure they are continuous and derivable in their domain. Else the learning algorithm (gradient descent) might not feedback correctly the error back into first layers.

You should normalize each input to either a range such as [-1,+1] or [-std,+std] using zscore. Therefore, the values of your inputs will have a similar weight in the decision function.

You do not specify the targets of your outputs, if they are discrete or floating point.

I wonder, as FFNN are supervised, with what data are you training your algorithm?

Upvotes: 1

Related Questions