CODEWITHSUNDEEP
machine-learningtensorflow
Ahmed Mohamed
Ahmed Mohamed

Reputation: 231

Tensorflow MNIST using convolution parameters

I dont understand why in official documentation he use the bias_variable of size 32, as i know the bias num is equal to num of neurons in the layer and in this case the number of neurons in the first layer is equal to 28 because the image pixels = 28 and he use the padding = "SAME", why it is equal 32 not 28?

Upvotes: 0

Views: 73

Answers (2)

Salvador Dali
Salvador Dali

Reputation: 222511

They use bias of size 32 to be compatible with the weights:

W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])

They use weights in conv2d function tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME').

tf.nn.conv2d() tells that the second parameter represent your filter and consists of [filter_height, filter_width, in_channels, out_channels]. So [5, 5, 1, 32] means that your in_channels is 1: you have a greyscale image, so no surprises here.

32 means that during our learning phase, the network will try to learn 32 different kernels which will be used during the prediction. You can change this number to any other number as it is a hyperparameter that you can tune.

Upvotes: 0

Luis Leal
Luis Leal

Reputation: 3514

Remember that mnist is using convolutional networks, not conventional neural networks and hence, you are dealing with convolutions(not neurons) and in this example , and in convolutions you commonly use a bias for every output channel, and this example uses 32 output channels in the first convolution layer and that gives you 32 biases.

Upvotes: 1

Related Questions