CODEWITHSUNDEEP
machine-learningneural-networkconvolutiondeep-learningconv-neural-network
user2202420
user2202420

Reputation: 159

How to find dynamically the depth of a network in Convolutional Neural Network

I was looking for an automatic way to decide how many layers should I apply to my network depends on data and computer configuration. I searched in web, but I could not find anything. Maybe my keywords or looking ways are wrong.

Do you have any idea?

Upvotes: 4

Views: 649

Answers (1)

Flavio Ferrara
Flavio Ferrara

Reputation: 1644

The number of layers, or depth, of a neural network is one of its hyperparameters.

This means that it is a quantity that can not be learned from the data, but you should choose it before trying to fit your dataset. According to Bengio,

We define a hyper- parameter for a learning algorithm A as a variable to be set prior to the actual application of A to the data, one that is not directly selected by the learning algo- rithm itself.

There are three main approaches to find out the optimal value for an hyperparameter. The first two are well explained in the paper I linked.

  • Manual search. Using well-known black magic, the researcher choose the optimal value through try-and-error.
  • Automatic search. The researcher relies on an automated routine in order to speed up the search.
  • Bayesian optimization.

More specifically, adding more layers to a deep neural network is likely to improve the performance (reduce generalization error), up to a certain number when it overfits the training data.

So, in practice, you should train your ConvNet with, say, 4 layers, try adding one hidden layer and train again, until you see some overfitting. Of course, some strong regularization techniques (such as dropout) is required.

Upvotes: 1

Related Questions