CODEWITHSUNDEEP
machine-learningneural-network
pjk
pjk

Reputation: 149

How to speed up the training in neural network when mini-batch training is used?

Can anyone give me some ideas on possible techniques to speed up the training process of multilayer artificial neural network if the training involves mini-batch?

So far, I understand that stochastic training probably leads to a faster convergence but, if we have to use mini-batch training, is there any way to make the convergence faster?

(Some pointers to relevant papers will also help!)

Thank you!

Upvotes: 2

Views: 1419

Answers (1)

rahulm
rahulm

Reputation: 734

In terms of general tips, you may want to be using a GPU when writing your neural network; they lead to huge improvements in speed and performance. See http://web.stanford.edu/~acoates/papers/CoatesHuvalWangWuNgCatanzaro_icml2013.pdf

In terms of machine learning, better weight initialization leads to faster convergence. One method of doing this is unsupervised feature learning. See http://web.stanford.edu/~acoates/papers/CoatesLeeNg_nips2010_dlwkshp_singlelayer.pdf

Upvotes: 2

Related Questions