CODEWITHSUNDEEP
pythonnumpymachine-learningscipymathematical-optimization
James Atwood
James Atwood

Reputation: 4329

Element-wise constraints in scipy.optimize.minimize

I'm using scipy.optimize.minimize's COBYLA method to find a matrix of parameters for a categorical distribution. I need to impose the constraint that each parameter is greater than zero, and that the sum of the rows of the parameter matrix is a column of ones.

It's not clear to me how to implement this in scipy.minimize, because the constraints are checked for non-negativity rather than truth. The minimization raises an exception if I just pass the arrays as the constraint.

Does anyone know how to go about implementing these kinds of constraints?

Upvotes: 8

Views: 9511

Answers (2)

ali_m
ali_m

Reputation: 74182

The first constraint x > 0 can be expressed very simply:

{'type':'ineq', 'fun': lambda x: x}

The second constraint is an equality constraint, which COBYLA doesn't natively support. However, you could express it as two separate inequality constraints instead:

{'type':'ineq', 'fun': lambda x: np.sum(x, 0) - 1}  # row sum >= 1
{'type':'ineq', 'fun': lambda x: 1 - np.sum(x, 0)}  # row sum <= 1

Otherwise you could try SLSQP instead, which does support equality constraints.

Upvotes: 11

MB-F
MB-F

Reputation: 23637

You need equality constraints that enforce np.sum(x, 1) == 1 and inequality constraints for x >= 0.

However, the COBYLA method can only handle inequality constraints, as explained in the documentation of minimize (see the section that explains the constraints argument). Instead, you can use Sequential Least SQuares Programming (SLSQP), which supports both types of constraints. The minimize function should automatically choose the correct solver for you, based on the constraints you specify.

The constraints you need can be implemented like this:

def ineq_constraint(x):
    """constrain all elements of x to be >= 0"""
    return x

def eq_constraint(x):
    """constrain the sum of all rows to be equal to 1"""
    return np.sum(x, 1) - 1


constraints = [{'type': 'ineq', 'fun': ineq_constraint},
               {'type': 'eq', 'fun': eq_constraint}]

result = minimize(objective_function, x0, constraints=constraints)

Upvotes: 8

Related Questions