CODEWITHSUNDEEP
memorymachine-learningneural-networkdeep-learningconv-neural-network
Dang Manh Truong
Dang Manh Truong

Reputation: 656

CS231n: Total memory of VGGnet

I'm reading this CS231n tutorial, about convolutional neural networks. They give an example about VGGNet:

http://cs231n.github.io/convolutional-networks/

VGGNet in detail. Lets break down the VGGNet in more detail as a case study. The whole VGGNet is composed of CONV layers that perform 3x3 convolutions with stride 1 and pad 1, and of POOL layers that perform 2x2 max pooling with stride 2 (and no padding). We can write out the size of the representation at each step of the processing and keep track of both the representation size and the total number of weights:

Then they give a detailed calculation of the network structure: enter image description here

But the thing is, for total memory, the tutorial gives the result of 24M, but when I calculated it I only got about 15M ! I simply added all of the memories:

>>> 224*224*(3+64*2)+112*112*(64+128*2)+56*56*(128+256*3)+28*28*(256+512*3)+14*14*(512*4)+7*7*512+4096+4096+1000
15237608

Please help me.

Upvotes: 0

Views: 2037

Answers (1)

Maxim
Maxim

Reputation: 53758

Nice catch! Your calculation is correct, total memory of VGG representation is indeed

15.2M * 4 bytes ~= 61Mb

In fact, this error has been reported long time ago, but unfortunately CS231n staff doesn't spend too much time on website maintenance...

However, note that if you code VGG network in any framework (Caffe, Tensorflow, etc), the total model size will include the parameters and this part is much larger, as the authors also show in their calculations (which seems right).

Upvotes: 2

Related Questions