CODEWITHSUNDEEP
deep-learningconv-neural-network
torayeff
torayeff

Reputation: 9702

Calculating the number of weights in Convolutional Neural Network using Parameter Sharing

While reading a book Machine Learning: a Probabilistic Perspective by Murphy and this article by Mike O'Neill I have encountered some calculations about the number of weights in Convolutional Neural Network which I want to understand. The architecture of the network is like this:

enter image description here

And this is the explanation from the above article:

Layer #2 is also a convolutional layer, but with 50 feature maps. Each feature map is 5x5, and each unit in the feature maps is a 5x5 convolutional kernel of corresponding areas of all 6 of the feature maps of the previous layers, each of which is a 13x13 feature map. There are therefore 5x5x50 = 1250 neurons in Layer #2, (5x5+1)x6x50 = 7800 weights, and 1250x26 = 32500 connections.

The calculation of the number of weights, (5x5+1)x6x50 = 7800, seems strange for me. Shouldn't be the actual calculation like this: (5x5x6+1)x50 = 7550 according to the parameter sharing explained here.

My argument is as follows: We have 50 filters of size 5x5x6 and 1 bias for each filter, hence the total number of weights is (5x5x6+1)x50=7550. And this is Pytorch code which verifies this:

import torch
import torch.nn as nn

model = nn.Conv2d(in_channels=6, out_channels=50, kernel_size=5, stride=2)
params_count = sum(param.numel() for param in model.parameters() if param.requires_grad)
print(params_count) # 7550

Can anyone explain this and which one is correct?

Upvotes: 0

Views: 2438

Answers (1)

v09
v09

Reputation: 890

My calculations:

Layer-1 depth is 6, kernel : 5*5
Layer-2 depth is 50 , kernel : 5*5

Total number of neurons #Layer-2 : 5*5*50 = 1250

Total number of weights would be: 5*5*50*6 = 7500

Finally, bias for #Layer-2 = 50 (depth is 50)

I agree with you : Total weights must be 7550.

Upvotes: 1

Related Questions