Make Your Own Neural Network Back Propagation

Question

I'm reading through the Make Your Own Neural Network book and in the chapter where the author describes about the back propagation, I find myself confused. I would like to relate the way the author explains with an example where he shows a 2 node, 3 layer network sh shown in the image below:

If I construct the Matrix representation of the above Neural Network for back propagation, it would look like this:

Where W^T_{hidden_output} is a Matrix transpose of the input weights, thus the Matrix representation in details is:

So if I want to now calculate the hidden Errors (e_{1_hidden} and e_{2_hidden}), I have the following:

e_{1_hidden} = W₁₁ * e₁ + W₁₂ * e₂

e_{2_hidden} = W₂₁ * e₁ + W₂₂ * e₂

But if I apply the values as given in the example., where e₁ = 1.5 and e₂ = 0.5, I do not get the e_{1_hidden} = 0.7 and e_{2_hidden} = 1.3

Where am I going wrong with my understanding / calculation? Any help?

Answer 1

You are describing the error back propagation based on simply multiplying by link weights, whereas the picture splits the error in proportion to the link weights. Both differences are described e.g. on this webpage:

In the picture, you see that the error is split in proportion to the link weights, e.g. e₁ = 1.5 is split into 0.6 and 0.9 according to the weights of W₁₁ = 1.0 and W₂₁ = 3.0. (Note further, that the subscripts of the weights are also wrong in the picture because there is only W₁₁ and all others are denoted by W₁₂ ...)

This split up error is then added to the final error of the hidden layer, e.g.:

e_{hidden 1} = 0.6 + 0.1 = 0.7.