CODEWITHSUNDEEP
optimizationmachine-learningneural-networkdeep-learning
Ahmad Karrabi
Ahmad Karrabi

Reputation: 61

best neural network optimization algorithms

i'm looking for 4 best algorithms in neural network optimization. i need the names and any description ,learning method and topology is very helping me. i think multilayer perceptron (MLP) is one of them but i'm not sure.

thank you all.

Upvotes: 0

Views: 4063

Answers (2)

Tshilidzi Mudau
Tshilidzi Mudau

Reputation: 7899

First of all, I agree that the question is indeed too broad, however, I would like to add some resorces I personaly found to be usefull in helping me answer this question. In doing so, I hope that this resources will help someone else in as much as they helped me. Here they are:

  • Sebastian Ruder Mod Yu Zeng's blog post.
  • Anish Singh Walia's discussion on this topic.
  • Chapter 8 of The deep learning book by an Goodfellow, Yoshua Bengio and Aaron Courville.
  • This post from the Morning paper by Adrian Colyer.

I hope this helps someone.

Upvotes: 0

Marcin Możejko
Marcin Możejko

Reputation: 40516

Your question is really wide. It's hard to answer because different optimizers have their pros and drawbacks. I would say that among the best today's optimizers you will find ADAM, RMSPROP, ADAGRAD and SGD with Nestov momentum. You can read about these methods (as well as others) here.

Upvotes: 5

Related Questions