CODEWITHSUNDEEP
neural-networkbias-neuron
Bing Bang
Bing Bang

Reputation: 534

A concern about the bias node in ann

OK so I get the idea of the bias node. It moves the transfer function curve horizontally to make it better fit the data. The problem I see is that the bias node weight value is computed just like any other weights. Is this right? Should the bias weight be computed in some other manner? And also shouldn't there be another bias value to move the transfer function up and down? Like this: f(x1+x2...+b1)+b2. I have no idea on how you would compute b2. Any Ideas?

Upvotes: 0

Views: 95

Answers (1)

Gaurav
Gaurav

Reputation: 1587

For your first question, computing the bias node weight value like any other weight, i.e. by using back-propagation is the most sound way of training and works well. There are other methods of training, but empirically back-propagation one is better.

The answer to your second question is that f(x1+x2...+b1)+b2 doesn't make sense. You have a one-dimensional input to your activation function (f). The y-axis plots the output. Biases don't change the shape of your function curve. For every intercept the function makes on the y-axis, you will have a unique intercept on X-axis. So the X-axis intercept is enough to describe the curve. Just like y=mx+c is enough to describe any line.

f(x1+x2...+b1)+b2 can be decomposed into f(x'1+x'2...+b'1) for all activation functions where weights x'1 , x'2... and b'1 can incorporate the vertical moment of function on the graph and this becomes your new activation function.

Upvotes: 1

Related Questions