CODEWITHSUNDEEP
javaoptimizationparametersmachine-learning
Daniel Gerber
Daniel Gerber

Reputation: 3470

Realworld parameter optimization

I'm in the need to do parameter optimization for my latest research project. I have an algorithm which has currently 5 parameters (four double [0,1] and one nominal with 3 values). The algorithm uses those parameters to calculate some stuff and afterwards I calculate the Precision, Recall & FMeasure. A single run takes about 1,8s. Currently I'm going through each parameter with a 0.1 step size which shows me approximately where the global maxima is. But I want to find the precise global maximum. I've looked into Gradient Descent but I don't really know how to apply this to my algorithm (if it's even possible). Could anybody please guide me a little how I would implement such an algorithm since I'm very new to this kind of work.

Cheers, Daniel

Upvotes: 4

Views: 1064

Answers (2)

陳俊方
陳俊方

Reputation: 1

Generally optimization requires a function to be used. If you have a real function, you can use it. However, in general practical processes or machines, there is no actual real function. (Unless you use a simulation function) So, you need to build an AI virtual model to replace this function, which is called "surrogate model optimization".

to create a first ai model, need collect some x-y pair dataset, if you have real function ,then random sampling it. or you need to operation real process or equipment to get parameter data (x), and the measure the quality (y).

so, if you build a ai model, or use real function. maybe you can search "scipy.optimize" some information. https://docs.scipy.org/doc/scipy/reference/optimize.html

Generally used minimize(method=’BFGS’) if you have parameter bounds require, maybe used minimize(method=’L-BFGS-B’).

the optimize result is a suggestion parameter dataset, you need to put in the ai model or real function, to get measure quality (y) result. to compare the correct or not. the optimization are the multiple loops process, to find the final optimize point.

Upvotes: 0

Don Reba
Don Reba

Reputation: 14041

You can certainly do better than a grid search.

Before applying an algorithm like gradient descent, you have to be sure that your parameter space does not contain local maxima or that at least your starting point is close to the global maximum and your step size is appropriate enough to bring you to it.

In your case, I would recommend starting by drawing as many random samples as you can. This is a much better way of exploring the parameter space than a grid search. Once you collect enough data this way, you can use a mode-finding algorithm, such as mean shift or one of its faster derivatives, or go straight to optimization. Since you don't have the Jacobian of your parameter space, you could use the Broyden's method, which iteratively approximates it or a secant method, such as BFGS.

Also, see this related question: How can I adjust parameters for image processing algorithm in an efficient way?

Upvotes: 3

Related Questions