CODEWITHSUNDEEP
pythonnumpyfunctionscipyminimize
sedavidw
sedavidw

Reputation: 11691

Structure of inputs to scipy minimize function

I have inherited some code that is trying to minimize a function using scipy.optimize.minimize. I am having trouble understanding some of the inputs to the fun and jac arguments.

The call to minimize looks something like this:

result = minimize(func, jac=jac_func, args=(D_neg, D, C), method = 'TNC' ...other arguments)

func looks like the following:

def func(G, D_neg, D, C):
    #do stuff

jac_func has the following structure:

def jac_func(G, D_neg, D, C):
    #do stuff

What I don't understand is where the G input to func and jac_func is coming from. Is that somehow specified in the minimize function, or by the fact that the method is specified as TNC? I've tried to do some research into the structure of this optimization function but I'm having trouble finding the answer I need.

Upvotes: 46

Views: 78091

Answers (2)

cottontail
cottontail

Reputation: 23071

For illustration purposes, we can print how G changes as minimize iterates to the local minimum. For example, consider the following case where the objective is to minimize a quadratic function of the form f(x) = 2*x**2 - 4*x + 7 (which attains its minimum at x=1). Note that the parameters of the function (2, -4 and 7) were supplied as arguments to obj_func below.

The initial guess has to be supplied to start the algorithm.

As the output shows, starting from the initial value of 10, the function variable G descends to the minimizer.

from scipy.optimize import minimize

def obj_func(G, a, b, c):
    print(G)
    return a*G**2 + b*G + c

initial_guess = 10
a, b, c = 2, -4, 7

result = minimize(obj_func, x0=initial_guess, args=(a, b, c))

print(f"\nMinimizer = {result.x}")
print(f"  Minimum = {result.fun}")

which outputs the following:

[10.]
[10.00000001]
[8.99]
[8.99000001]
[6.82085113]
[6.82085114]
[1.62275043]
[1.62275045]
[1.]
[1.00000001]

Minimizer = [1.]
  Minimum = 5.0

Another example: Consider a two-variable function of the form f(x, y) = (x - 1)**2 + (y - 2)**2. The function attains its minimum at (x, y) = (1, 2) (and the minimum is 0).

Then starting from an initial point of (x, y) = (0, 3), the function converges to the minimum as in the following.

def obj_func(variable, x_offset, y_offset):
    x, y = variable
    print(f"x={x:.3f}, y={y:.3f}")
    return (x - x_offset)**2 + (y - y_offset)**2

initial_guess = [0, 3]
result = minimize(obj_func, initial_guess, args=(1, 2))

This prints the following which shows that the variables converge to the minimizer.

x=0.000, y=3.000
x=0.000, y=3.000
x=0.000, y=3.000
x=0.714, y=2.286
x=0.714, y=2.286
x=0.714, y=2.286
x=1.000, y=2.000
x=1.000, y=2.000
x=1.000, y=2.000

An important note about minimize is that the initial guess has to be an educated one especially if the objective function is complex, otherwise the algorithm may not converge. I find that to be a major source of unsuccessful optimization run.

Upvotes: 0

lmjohns3
lmjohns3

Reputation: 7592

The short answer is that G is maintained by the optimizer as part of the minimization process, while the (D_neg, D, and C) arguments are passed in as-is from the args tuple.

By default, scipy.optimize.minimize takes a function fun(x) that accepts one argument x (which might be an array or the like) and returns a scalar. scipy.optimize.minimize then finds an argument value xp such that fun(xp) is less than fun(x) for other values of x. The optimizer is responsible for creating values of x and passing them to fun for evaluation.

But what if you happen to have a function fun(x, y) that has some additional parameter y that needs to be passed in separately (but is considered a constant for the purposes of the optimization)? This is what the args tuple is for. The documentation tries to explain how the args tuple is used, but it can be a little hard to parse:

args: tuple, optional

Extra arguments passed to the objective function and its derivatives (Jacobian, Hessian).

Effectively, scipy.optimize.minimize will pass whatever is in args as the remainder of the arguments to fun, using the asterisk arguments notation: the function is then called as fun(x, *args) during optimization. The x portion is passed in by the optimizer, and the args tuple is given as the remaining arguments.

So, in your code, the value of the G element is maintained by the optimizer while evaluating possible values of G, and the (D_neg, D, C) tuple is passed in as-is.

Upvotes: 66

Related Questions