CODEWITHSUNDEEP
neural-network
Razorocean
Razorocean

Reputation: 398

Sample sizes in neural networks

I've seen that the general rule of sample sizes in neural networks is 10 times the 'number of weights'. So for example, if we have a NxD input, one layer size M, and output layer size K, is the 'number of weights' 2 because of the two W matrices? Or is it DM + MK? Thank you in advance for your help.

Upvotes: 1

Views: 192

Answers (1)

Ali
Ali

Reputation: 1635

The actual number of weights should be DM + MK + M + K where DM represents the number of weights for all unit in the hidden layer and M is for the bias terms in the hidden layer and in the same way, MK and K represent number of wights and bias terms in the output layer respectively.

Upvotes: 1

Related Questions