CODEWITHSUNDEEP
machine-learningneural-network
haruishi
haruishi

Reputation: 23

How to calculate the amount of connections in neural network

I have exactly this scenario and I need to know how many connections this set has. I searched in several places and I'm not sure of the answer. That is, I do not know how to calculate the number of connections on my network, this is still unclear to me. What I have is exactly as follows:

** Having bias in all but least of the input

I would calculate this as follows: ((784 * 400) + bias) + ((400 * 200) + bias) + ((200 * 10) + bias) = XXX

I do not know if this is correct. I need help figuring out how to solve this, and if it's not just something mathematical, what's the theory to do this calculation?

Thank you.

Upvotes: 2

Views: 6424

Answers (2)

Mahdyfo
Mahdyfo

Reputation: 1173

The maximum number of weights (connections or genes) in a neural network that the inputs can connect to the outputs and the hidden neurons can connect to each other, is:

(inputs * outputs) + 
(inputs * hiddens) +
(hiddens * outputs) +
[hiddens * (hiddens - 1)]

In case of not having any to-future feedbacks, meaning that the neurons can be only connected to the previous ones (for memory-less networks), it will be:

(inputs * outputs) + 
(inputs * hiddens) +
(hiddens * outputs) +
[hiddens * (hiddens - 1)] / 2

In case of old fashion static multi-layer neural networks:

(inputs * layer1-hiddens) +
(layer1-hiddens * layer2-hiddens) +
(layer2-hiddens * layer3-hiddens) +
...
(last-layer-hiddens * outputs]

We considered the bias neurons to be among the input neurons in these examples.

Upvotes: 0

enumaris
enumaris

Reputation: 1938

Your calculation is correct for total number of weights. When you have n neurons connected to m neurons, the number connections between neurons is n*m. You can see this by drawing a small graph, say 3 neurons connected to 4 neurons. You will see that there's 12 connections between the two layers. So if you want connections rather than weights, just drop the '+bias' parts of your equation.

If you want total weights, then the number is simply (n*m+m) since you get 1 bias weight for each of the m neurons in the second layer.

Total connections in that neural network: (784*400)+(400*200)+(200*10)

Total weights in that neural network: (784*400+400)+(400*200+200)+(200*10+10)

Upvotes: 3

Related Questions