CODEWITHSUNDEEP
conv-neural-network
cerebrou
cerebrou

Reputation: 5540

Does the input size to CNN change depending on the batch size?

Usually, the input to a convolutional neural network (CNN) is described by an image with given width*height*channels. Will the number of input nodes be different for different numbers of a batch size, as well? Meaning, will the number of input nodes become batch_size*width*height*channels?

Upvotes: 1

Views: 1986

Answers (1)

jla
jla

Reputation: 4761

From this excellent answer batch size defines number of samples that going to be propagated through the network. Batch size does not have an effect on the network's architecture, including quantity of inputs.

Let's say you have 1,000 RGB images with size 32x32, and you set the batch size to be 100. Your convolutional network should have an input shape of 32x32x3. To train the network the training algorithm picks a sample of 100 images out of the total 1,000, and trains the network on each individual image in that subset. That is your 'batch'. The network architecture doesn't care that your subset (batch) has 100, 200 or 1,000 images, it only cares about the shape of a single image, because that's all it sees at a time. When the network has been trained on all 100 images then it has completed one epoch, and the network parameters are updated.

The training algorithm will pick a different batch of images for each epoch, but the above always holds true: the network only sees a single image at a time, so the image shape must match the input layer shape, with no regard for how many images in that particular batch.

As to why we have batches instead of just training on the entire set (ie setting batch size to 100% and training for one epoch), with GPUs it makes training much faster, and also means the parameters are updated less often while training, giving a smoother and more reliable convergence.

Upvotes: 2

Related Questions