CODEWITHSUNDEEP
machine-learningneural-network
Louis
Louis

Reputation: 4210

Number of hidden layers in a neural network model

Would someone be able to explain to me or point me to some resources of why (or situations where) more than one hidden layer would be necessary or useful in a neural network?

Upvotes: 9

Views: 3250

Answers (3)

Ofri Raviv
Ofri Raviv

Reputation: 24823

  1. That's more similar to the way the brain works (which might not necessarily be a computational advantage, but a lot of people are researching NN to gain insight about the way the mind works, rather than to solve real world problems.
  2. Its easier to achieve some kinds of invariance using more layers. For example, an image classifier that works regardless of where in the image the object is found, or the object's size. see Bouvrie, J. , L. Rosasco, and T. Poggio. "On Invariance in Hierarchical Models". Advances in Neural Information Processing Systems (NIPS) 22, 2009.

Upvotes: 4

Max Shawabkeh
Max Shawabkeh

Reputation: 38663

Basically more layers allow more functions to be represented. The standard book for AI courses, "Artificial Intelligence, A Modern Approach" by Russell and Norvig, goes into some detail of why multiple layers matter in Chapter 20.

One important point is that with a sufficiently large single hidden layer, you can represent every continuous function, but you will need at least 2 layers to be able to represent every discontinuous function.

In practice, though, a single layer is enough at least 99% of the time.

Upvotes: 6

Amber
Amber

Reputation: 527328

Each layer effectively raises the potential "complexity" of adaptation in an exponential fashion (as opposed to a multiplicative fashion of adding more nodes to a single layer).

Upvotes: 2

Related Questions