How to determine the threshold for neuron firings in neural networks?

Question

I have a simple task to classify people by their height and hair length to either MAN or WOMAN category using a neural network. Also teach it the pattern with some examples and then use it to classify on its own.

I have a basic understanding of neural networks but would really need some help here.

I know that each neuron divides the area to two subareas, basically that is why P = w0 + w1*x1 + w2*x2 + ... + wn*xn is being used here (weights are just moving the line if we consider geometric representation).

I do understand that each epoche should modify the weights to get closer to correct result, yet I have never program it and I am hopeless about how to start.

How should I proceed, meaning: How can I determine the threshold and how should I deal with the inputs?

It is not a homework rather than task for the ones who were interested. I am and I would like to understand it.

Answer 1

Since you said:

I know that each neuron divides the area to two subareas

&

weights are just moving the line if we consider geometric representation

I think you want to use perseptron or ADALINE neural networks. These neural networks can just classify linear separable patterns. since your input data is complicated, It's better to use a Multi layer Non-Linear Neural network. (my suggestion is a two layer neural network with tanh activation function) . For training these network you should use back propagation algorithm.

For answering to

how should I deal with the inputs?

I need to know more details about the inputs( Like: are they just height and hair length or there is more, what is their range and your resolution and etc.)

If you're dealing with just height and hair length I suggest that divide heights and length in some classes (for example 160cm-165cm, 165cm-170cm & etc.) and for each one of these classes set an On/Off input neuron. then put a hidden layer after all classes related to heights and another hidden layer after all classes related to hair length (tanh activation function). Number of neurons in these two hidden layer is determined based on number of training cases. then take these two hidden layer output and send them to an aggregation layer with 1 output neuron.

Answer 2

Looks like you are dealing with a simple Perceptron with a threshold activation function. Have a look at this question. Since you ARE using a bias neuron (w0), you would set the threshold to 0.

You then simply take the output of your network and compare it to 0, so you would e.g. output class 1 if x < 0 and class 2 if x > 0. You could model the case x=0 as "indistinct".

For learning the weights you need to apply the Delta Learning Rule which can be implemented very easily. But be careful: a perceptron with a simple threshold activation function can only be correct if your data are linearly separable. If you have more complex data you will need a Multilayer Perceptron and a nonlinear activation function like the Logistic Sigmoid Function.

Have a look at Geoffrey Hintons Coursera Course, Lecture 2 for details.

Answer 3

I've been working with machine learning lately (but I'm not an expert) but you should look at the Accord.NET framework. It contains all the common machine learning algorithme out of the box. So it's easy to take an existing samples and modify it instead of starting from scratch. Also, the developper of the framework is very helpful in the forum available on the same page.

With the available samples, you may also discover something better than neural network like the Kernel Support Vector Machine. If you stick to the neural network, have fun modifying all the different variables and by tryout and error you will understand how it work.

Have fun!