CODEWITHSUNDEEP
pythonnumpyscipy
Bram Zijlstra
Bram Zijlstra

Reputation: 390

Using scipy minimize with constraint on one parameter

I am using a scipy.minimize function, where I'd like to have one parameter only searching for options with two decimals.

def cost(parameters,input,target):
    from sklearn.metrics import mean_squared_error
    output = self.model(parameters = parameters,input = input)
    cost = mean_squared_error(target.flatten(), output.flatten())

    return cost

parameters = [1, 1] # initial parameters
res = minimize(fun=cost, x0=parameters,args=(input,target)
model_parameters = res.x

Here self.model is a function that performs some matrix manipulation based on the parameters. Input and target are two matrices. The function works the way I want to, except I would like to have parameter[1] to have a constraint. Ideally I'd just like to give an numpy array, like np.arange(0,10,0.01). Is this possible?

Upvotes: 0

Views: 225

Answers (1)

sascha
sascha

Reputation: 33522

In general this is very hard to do as smoothness is one of the core-assumptions of those optimizers.

Problems where some variables are discrete and some are not are hard and usually tackled either by mixed-integer optimization (working good for MI-linear-programming, quite okay for MI-convex-programming although there are less good solvers) or global-optimization (usually derivative-free).

Depending on your task-details, i recommend decomposing the problem:

  • outer-loop for np.arange(0,10,0.01)-like fixing of variable
  • inner-loop for optimizing, where this variable is fixed
  • return the model with the best objective (with status=success)

This will effect in N inner-optimizations, where N=state-space of your to fix-var.

Depending on your task/data, it might be a good idea to traverse the fixing-space monotonically (like using np's arange) and use the solution of iteration i as initial-point for the problem i+1 (potentially less iterations needed if guess is good). But this is probably not relevant here, see next part.

If you really got 2 parameters, like indicated, this decomposition leads to an inner-problem with only 1 variable. Then, don't use minimize, use minimize_scalar (faster and more robust; does not need an initial-point).

Upvotes: 1

Related Questions