CODEWITHSUNDEEP
tensorflowmachine-learningneural-networksoftmax
Wayde Herman
Wayde Herman

Reputation: 71

Finding Global minima

I have a neural network and am struggling to find the global minima. I would like to know the right approach to finding it. I know I need to change the hyperparameters but because you use random weights, do I try using the same hyperparameters multiple times? How many different combinations do I try before I give up?

Any practical advice would be appreciated.

Upvotes: 1

Views: 1021

Answers (2)

user9633488
user9633488

Reputation:

This is the major drawback of gradient descent based optimization. They get stuck in local minima all the time and there is no way of guaranteeing a global minima solution.

You can use terms like momentum to help you get over some local minima. I would suggest trying some more advanced optimization technique. These will help you a bit because they select and update some of these hyperparameters automatically.

You can also look at this post right here which compares several optimization algorithms.

Upvotes: 1

Bwebb
Bwebb

Reputation: 685

I dont think there is a way to find a global minima for a NN with gradient descent (assuming thats what your using). I think you can find it with an SVM though.

Im not a machine learning expert so i could be wrong.

Upvotes: 0

Related Questions